Skip to content

Commit d7bf338

Browse files
chore(docs): document how code is generated (#18880)
* chore(docs): document how code is generated Credit: Diego's https://docs.google.com/document/d/1fcg-Gg6sq8JSHzo-baHeAeXDkkCRDiabDwjmIke1lwU/edit?usp=sharing * Update README.md * Update README.md * Moved the source code comment to YAML * comment * Update README.md Co-authored-by: Diego Marquez <diegomarquezp@google.com> --------- Co-authored-by: Diego Marquez <diegomarquezp@google.com>
1 parent fd786a7 commit d7bf338

File tree

2 files changed

+41
-2
lines changed

2 files changed

+41
-2
lines changed

.github/workflows/codegen.yaml

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ on:
77
name: codegen
88
jobs:
99
discovery:
10+
# As of November 2023, it returns 269 service names.
1011
uses: googleapis/discovery-artifact-manager/.github/workflows/list-services.yml@master
1112
total_service_size_check:
1213
runs-on: ubuntu-20.04
@@ -26,6 +27,9 @@ jobs:
2627
runs-on: ubuntu-20.04
2728
needs: discovery
2829
outputs:
30+
# As of November 2023, the output of batch job is a 3-element array, which contains a chunk of 100 service names
31+
# at the index 0, a chunk of other 100 service names at the index 1, and a chunk of remaining 69 service names
32+
# at the index 2.
2933
services: ${{ steps.chunk.outputs.result }}
3034
steps:
3135
- uses: actions/github-script@v5
@@ -48,7 +52,10 @@ jobs:
4852
uses: googleapis/google-api-java-client-services/.github/workflows/generate.yaml@main
4953
needs: batch
5054
secrets: inherit
51-
# The number of batch is implicitly decided by the hour of the day
55+
# The size of the batch is implicitly decided by the hour of the day.
56+
# For example, a job starting at "1:30" uses the chunk at the index 1 in the array.
57+
# If you manually invoke the workflow other hours than 0:00 - 2:59 and the array's length is 3,
58+
# this job is skipped because there's no element at the index in the array.
5259
if: ${{!!fromJson(needs.batch.outputs.services).batches[fromJson(needs.batch.outputs.services).hour]}}
5360
with:
5461
services: ${{toJson(fromJson(needs.batch.outputs.services).batches[fromJson(needs.batch.outputs.services).hour])}}

README.md

Lines changed: 33 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -262,7 +262,39 @@ see the [project's README](https://github.com/googleapis/google-auth-library-jav
262262
for how to use credentials with google-http-client and
263263
[javadoc](https://cloud.google.com/java/docs/reference/google-auth-library/latest/overview) for more details.
264264

265-
## Generating the API clients
265+
## How the code is updated and published
266+
267+
When a change is made in the API definitions, the following events happens:
268+
269+
1. [The discovery-artifact-manager repository](https://github.com/googleapis/discovery-artifact-manager) has
270+
[update-discoveries job](https://github.com/googleapis/discovery-artifact-manager/blob/master/.github/workflows/update-disco.yml)
271+
that copies the definition files from https://discovery.googleapis.com/discovery/v1/apis to
272+
the repository.
273+
2. This google-api-java-client-services repository has ([codegen workflow](
274+
https://github.com/googleapis/google-api-java-client-services/blob/main/.github/workflows/codegen.yaml)).
275+
This daily workflow has the following jobs:
276+
- **discovery**: It uses the discovery-artifact-manager repository's [List discovery services workflow](
277+
https://github.com/googleapis/discovery-artifact-manager/blob/master/.github/workflows/list-services.yml)
278+
to list the service names.
279+
Example values in the returned array include "storage" (from `discoveries/storage.v1.json`) and "admin"
280+
(from `discoveries/admin.datatransfer_v1.json`).
281+
- **total_service_size_check**: This job ensures the size of the service name array does not exceed 300
282+
(00:30 to 02:30 implies 3 batches of size 100); otherwise it fails.
283+
As of November 2023, the size of the service name array is 269.
284+
- **batch** This job splits the service names into chunks of maximum 100 and returns an array containing the chunks.
285+
This is to avoid one job that is too long to finish and also to avoid hitting the maximum parallel job limit of Github actions (256).
286+
- **generate**: At the end, this job runs the code generator for each service in the chunk whose array index
287+
corresponds to the hour of the day. For details, see [codegen.yaml](
288+
https://github.com/googleapis/google-api-java-client-services/blob/main/.github/workflows/codegen.yaml).
289+
If there are code changes, this job creates pull requests
290+
(Example: [#18860](https://github.com/googleapis/google-api-java-client-services/pull/18860)).
291+
3. The yoshi-approver and merge-on-green bots merge the pull requests automatically.
292+
4. [The Kokoro job](http://fusion2/search?q=google-api-java-client-services%2Frelease&s=p)
293+
publishes the libraries in this repository.
294+
295+
## Generating the API clients locally
296+
297+
If you want to generate certain code locally for troubleshooting purpose, please follow these steps:
266298

267299
Generating the API clients requires git and Python 3.6.
268300

0 commit comments

Comments
 (0)