Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 19 additions & 56 deletions .agents/skills/java-development/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,67 +1,30 @@
---
name: java-development
description: General guidance on Java development practices, building, testing, and style in the monorepo. Use this skill when working on Java code across the repository.
description: Workflow guidance for performing Java development tasks in this repository.
---

# Java Development Guide
# Java Development Workflow

This skill provides general guidelines for Java development inside the monorepo. It covers building, formatting, testing, and style conventions to ensure consistency across modules.
Use this skill when tasked with modifying Java code, fixing bugs, or adding features. It guides you through the typical development lifecycle in this repo.

## Workflow
## Workflow Steps

### 1. Building the Project
### Step 1: Understand the Scope
- Determine if you are working on **Core Components** (`sdk-platform-java`) or a specific **Client Library** (`java-<service>`).
- Review the root [AGENTS.md](../../../AGENTS.md) for the Style Guide and Dependency Management rules that apply repo-wide.

The repository uses Maven as its primary build system.
### Step 2: Make Changes and Build
- Apply your code changes.
- To build efficiently, use the recommended flags or scoped builds detailed in [AGENTS.md](../../../AGENTS.md#building-the-project).

* **Build All Modules**: To build all modules from the root of the repository, run:
```bash
mvn install -T 1C -P quick-build
```
> [!TIP]
> Use `-T 1C` to build modules in parallel (one thread per CPU core) and `-P quick-build` to skip unnecessary plugins for faster builds.
* **Build a Specific Module**: You can also run Maven commands within a specific module directory (e.g., `java-bigquery`) to build only that module.
### Step 3: Determine and Run Tests
- If working on a **Client Library**, run its unit tests (`mvn test` in the module directory) and check its README for integration test procedures.
- If working on **Core Components** (`sdk-platform-java`), you **must** follow the testing heuristics in [sdk-platform-java/DEVELOPMENT.md](../../../sdk-platform-java/DEVELOPMENT.md#testing-guide) to know which tests are required.
- If golden files need updating, follow the specific procedures referenced in [AGENTS.md](../../../AGENTS.md#8-specialized-development-guides).

### 2. Code Formatting
### Step 4: Format Code
- You **must** ensure the code is formatted before completing a task. Use the formatting commands listed in [AGENTS.md](../../../AGENTS.md#code-formatting).

Code formatting is enforced using the `fmt-maven-plugin`.

* **Check Formatting**: To check for formatting issues without modifying files, run:
```bash
mvn fmt:check -T 1C
```
* **Apply Formatting**: To automatically format the code according to the project style, run:
```bash
mvn fmt:format -T 1C
```
> [!TIP]
> To save time, run `mvn fmt:format` within the specific module directory you are working on, rather than at the root.
> [!NOTE]
> Always run `mvn fmt:format` before committing changes to avoid build failures due to formatting.

### 3. Testing Strategy

* **Unit Tests**: Traditional unit tests should be added for individual classes and methods. Run them using:
```bash
mvn test -T 1C
```
* **Integration Tests**: Many modules have integration tests that run against live services or emulators. These may require specific profiles or environment variables. Refer to the specific module's README for details.

### 4. Style Guide

Follow these general rules to maintain code quality and consistency:

1. **Minimize Visibility**: Default to the most restrictive access level possible. Avoid using `public` unless the class or method is intended to be part of the public API.
2. Avoid Fully Qualified Names: Use imports to keep class names short and readable, rather than using fully qualified names in the code.
3. **Avoid Obsolete APIs**: Do not call methods marked with `@ObsoleteApi` or `@Deprecated` unless there are no viable alternatives.
4. **Clean Diffs**: Avoid unnecessary formatting changes or whitespace modifications to keep diffs clean and easy to review.

### 5. Dependency Management

* **Version Bumps**: Try not to bump any external dependency versions unless there is a known security vulnerability (CVE) or a critical bug fix.
* **New Dependencies**: Avoid introducing new external dependencies. If a new dependency is required, provide a strong justification in the pull request.
* **Standard Library First**: Prefer to use features from the Java standard library, followed by existing dependencies in the project (preferably Google-managed dependencies).

### 6. Contribution Guidelines

* **Commit Messages**: Follow the [Conventional Commits](https://www.conventionalcommits.org/) specification. Include the module as the scope (e.g., `feat(spanner): ...`, `fix(bigquery): ...`).
* **Pull Requests**: All code changes must be submitted via a pull request and require review. Ensure you pull the latest changes from `main` and resolve any conflicts before submitting.
### Step 5: Verify and Document
- Verify that all tests pass and formatting is correct.
- Follow the Contribution Guidelines in [AGENTS.md](../../../AGENTS.md#7-contribution-guidelines) for creating commits and PRs.
76 changes: 76 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# Agents Guide for google-cloud-java

## 1. Overview
This repository, `google-cloud-java`, contains the Java client libraries for Google Cloud Platform services, as well as core components in `sdk-platform-java`.

## 2. Project Structure
The repository is a monorepo containing both generated and handwritten libraries, as well as core platform components and parent POMs.

### Core Modules
- **`sdk-platform-java/`**: Contains foundational components for building client libraries.
- **Note**: This directory has its own `GEMINI.md` file with detailed instructions specific to core development (GAPIC generator, GAX).
- Includes `gapic-generator-java` (the generator) and `gax-java` (Google API Extensions).
- **`google-auth-library-java/`**: The Google Auth Library for Java. This is a **handwritten** library used for authentication and credential management across all Google Cloud clients. It is a critical dependency for all client libraries.

### Parent Modules and BOMs
- **`google-cloud-pom-parent/`**: The top-level parent POM for all modules in the repository. It manages plugin versions and common configuration.
- **`google-cloud-jar-parent/`**: The parent POM for all client library JAR modules in the repository. It inherits from `google-cloud-pom-parent` and manages shared dependencies.
- **`gapic-libraries-bom/`**: The Bill of Materials (BOM) that manages versions of all client libraries to ensure compatibility when used together.
- **`java-shared-dependencies/`** (inside `sdk-platform-java`): Manages shared Maven dependencies for all Google Cloud Java client libraries to ensure consistency and avoid conflicts.

### Client Libraries (`java-<service>/`)
Directories starting with `java-` are client libraries for specific Google Cloud services.
- **Generated Clients**: The majority of these are automatically generated from service definitions (protos) using the GAPIC generator in `sdk-platform-java`.
- **Handwritten & Split Repositories**: Some major libraries are either entirely handwritten or are maintained as "split repos" (they have their own standalone repositories in the `googleapis` GitHub organization but are also managed here). When working on these, be aware that changes may need to be synchronized with their respective split repos. Key examples include:
- **BigQuery**: [java-bigquery](java-bigquery)
- **BigQuery Storage**: [java-bigquerystorage](java-bigquerystorage)
- **Spanner**: [java-spanner](java-spanner)
- **Spanner JDBC**: [java-spanner-jdbc](java-spanner-jdbc)
- **Storage**: [java-storage](java-storage)
- **Storage NIO**: [java-storage-nio](java-storage-nio)
- **Datastore**: [java-datastore](java-datastore)
- **Logging**: [java-logging](java-logging)
- **Logging Logback**: [java-logging-logback](java-logging-logback)

## 3. Getting Started
### Prerequisites
- Java 11+ (build targets Java 8 bytecode compatibility)
- Maven 3.0+
- Bazelisk (for integration tests in `sdk-platform-java`)

### Building the Project
- Build all modules: `mvn install` from root.
- Recommended for faster builds: `mvn install -T 1C -P quick-build`.
- Build specific module: Run `mvn` commands within the module directory.

### Code Formatting
- Check: `mvn fmt:check`
- Format: `mvn fmt:format`
- Tip: Run in specific module to save time.

## 4. Testing Strategy
- **Unit Tests**: `mvn test`.
- **Integration Tests**: Module specific, may require emulators or live services.
- **Core Components Testing**: Components should have adequate unit and integration tests to ensure coverage and correctness.

## 5. Style Guide
1. Minimize visibility scopes. Default to most restrictive access level.
2. Use short names over fully qualified names.
3. Avoid calling `@ObsoleteApi` or `@Deprecated` methods and classes.
4. Avoid unnecessary formatting changes to keep diffs clean.
5. Use `mvn` for everything other than the `test/integration` folder.

## 6. Dependency Management
- Do not bump external dependency versions unless for CVE or critical bug fix.
- Avoid introducing new external dependencies if possible. Prefer Java standard library, then opt for existing dependencies.

## 7. Contribution Guidelines
- **Commits**: Conventional Commits `<type>(<scope>): <description>`.
- **Pull Requests**: Submitted via PR, require review. Pull latest from main and resolve conflicts.
- **Testing**: All new logic should be accompanied by tests.

## 8. Specialized Development Guides
For development on core components, refer to the following guides in `sdk-platform-java`:
- **GAPIC Generator**: [sdk-platform-java/gapic-generator-java/DEVELOPMENT.md](sdk-platform-java/gapic-generator-java/DEVELOPMENT.md)
- **GAX**: [sdk-platform-java/gax-java/DEVELOPMENT.md](sdk-platform-java/gax-java/DEVELOPMENT.md)
- **Hermetic Build**: [sdk-platform-java/hermetic_build/DEVELOPMENT.md](sdk-platform-java/hermetic_build/DEVELOPMENT.md)
120 changes: 9 additions & 111 deletions sdk-platform-java/GEMINI.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,9 @@

## 1. Overview

This directory, sdk-platform-java, contains the foundational components for building Java client libraries for Google Cloud Platform services. It includes the GAPIC (Generated API Client) generator for Java, the GAX (Google API Extensions) runtime library, and other shared modules.
This directory, `sdk-platform-java`, contains the foundational components for building Java client libraries for Google Cloud Platform services. It includes the GAPIC (Generated API Client) generator for Java, the GAX (Google API Extensions) runtime library, and other shared modules.

For general project rules, style guides, dependency management, and contribution guidelines, please refer to the root [AGENTS.md](../AGENTS.md).

## 2. Project Structure

Expand All @@ -28,130 +30,26 @@ The repository is structured into several key modules:

* **`java-shared-dependencies`**: Manages shared Maven dependencies for all Google Cloud Java client libraries. This ensures consistency and helps avoid dependency conflicts.

## 3. Getting Started

### 3.1. Prerequisites

To build and work with this project, you will need to install:

* Java 11+
* Maven

### 3.2. Building the Project

The project uses Maven for its primary build system. To build all modules, run the following command from the root of the repository:

```sh
mvn install
```

### 3.3. Code Formatting

Code formatting is enforced using the `fmt-maven-plugin`. To check for formatting issues, run:

```sh
mvn fmt:check
```

To format the code, run:

```sh
mvn fmt:format
```

## 4. Testing

### 4.1. Testing Strategy

The repository employs a multi-layered testing strategy to ensure the quality and correctness of the generated code:

* **Unit Tests:** Traditional unit tests for individual classes and methods. See [unit testing best practices](https://engdoc.corp.google.com/eng/doc/devguide/testing/unit/best-practices.md?cl=head&polyglot=java).
* **Golden Unit Tests:** These tests generate code from test protos and compare the output to "golden" files, which are pre-approved versions of the generated code. These test cases exist inside the `gapic-generator-java` module. See [GrpcServiceStubClassComposerTest](https://github.com/googleapis/google-cloud-java/blob/main/sdk-platform-java/gapic-generator-java/src/test/java/com/google/api/generator/gapic/composer/grpc/GrpcServiceStubClassComposerTest.java) for an example of golden unit tests. The steps for golden unit tests include creating a test proto, loading the test proto, generating from the test proto, and comparing the result with a golden file.
* **Showcase Integration Tests:** These tests run the generated Showcase client against a local Showcase server to verify end-to-end functionality. This is the preferred way of testing integration tests.
* **Golden Integration Tests:** These tests generate full client libraries for real Google Cloud APIs and compare them to golden versions. This is an older test strategy and showcase testing is preferred.
## 3. Testing

Based on where the code changes are, we should add different tests, in general
For detailed testing strategy and heuristics on which tests to run based on code changes, please refer to [DEVELOPMENT.md](DEVELOPMENT.md).

* If the changes are in `gax` or `api-common` only, you _must_ add traditional unit tests, you _may_ add showcase integration tests if you are modifying a public setting.
* If the changes are in `gapic-generator-java` only, showcase integration tests are most likely not needed
* If they are in the `composer` module, you _must_ add golden unit tests, you _may_ add traditional unit tests for better coverage and easier testing.
* If they are in the `parser` module, you _should_ add traditional unit tests with a test proto if possible, you _may_ add golden unit tests to easily see the changes. For example, [routing_rule_parser_testing](https://github.com/googleapis/google-cloud-java/blob/main/sdk-platform-java/gapic-generator-java/src/test/proto/routing_rule_parser_testing.proto) that is only used for testing [RoutingRuleParser](https://github.com/googleapis/google-cloud-java/blob/main/sdk-platform-java/gapic-generator-java/src/main/java/com/google/api/generator/gapic/protoparser/RoutingRuleParser.java).
* If they are in `other modules(ast, model, writer etc.)`, you _must_ add traditional unit tests, you _may_ add golden unit tests to easily see the changes.
- If the changes are in both `gax` and `gapic-generator-java`, you _must_ add all test layers, including traditional unit tests, golden unit tests and showcase integration tests.

### 4.2. Running Unit Tests
### 3.1. Running Unit Tests

Unit tests can be tested via this command:

```sh
mvn test
```

### 4.3. Running Golden Integration Tests
### 3.2. Running Golden Integration Tests

Golden integration tests are run using Bazel. To run all golden integration tests, use the following command from the root of the repository:

```sh
bazelisk test //...
```

Specific integration tests can be run by specifying the target, for example:

```sh
bazelisk test //test/integration:redis
```

### 4.4. Updating Golden Files

If you make changes that affect the generated code, you will need to update the golden files. This can be done using the following command:

```sh
bazelisk run //test/integration:update_asset && bazelisk run //test/integration:update_credentials && bazelisk run //test/integration:update_iam && bazelisk run //test/integration:update_kms && bazelisk run //test/integration:update_pubsub && bazelisk run //test/integration:update_logging && bazelisk run //test/integration:update_redis && bazelisk run //test/integration:update_storage && bazelisk run //test/integration:update_library && bazelisk run //test/integration:update_compute && bazelisk run //test/integration:update_bigtable && bazelisk run //test/integration:update_apigeeconnect
```

### 4.5. Running Showcase Integration Tests

Showcase integration tests are run against a local server that implements the Showcase API. The `java-showcase/README.md` file provides detailed instructions on how to run these tests. The general steps are:

1. **Install the Showcase server:**

```sh
go install github.com/googleapis/gapic-showcase/cmd/gapic-showcase@latest
```

2. **Run the Showcase server:**

```sh
gapic-showcase run
```

3. **Run the integration tests:**

```sh
cd java-showcase
mvn verify -P enable-integration-tests
```

## 5. Style Guide

1. Minimize visibility scopes by defaulting to the most restrictive access level, avoiding the `public` modifier unless required.
2. Use short names over fully qualified names.
3. Avoid calling `@ObsoleteApi` or `@Deprecated` methods unless there are no alternatives.
4. Avoid unnecessary formatting changes to keep diffs clean.
5. Use `mvn` for everything other than the `test/integration` folder.

## 6. Dependency Management

- Try not to bump any external dependency version unless there is a known CVE (security or vulnerability issue) or a critical bug fix.
- Try to avoid introducing new external dependencies. If a new dependency is required, please state the reason.
- Prefer to use features from the Java standard library, then try to use features from any existing dependencies (preferably from Google managed dependencies).

## 7. Contribution Guidelines
### 3.3. Running Showcase Integration Tests

- **Commits:** Commit messages should follow the [Conventional Commits](https://www.conventionalcommits.org/)
specification. The format is `<type>: <description>`. The type should be one of the following: fix, feat,
build, chore, docs, test, or refactor.
- **Issues:** All significant changes should start with a GitHub issue.
- **Pull Requests:** All code changes must be submitted via a pull request and require review. Before creating a PR, always pull latest from main, merge main to local branch and resolve any conflicts.
- **Testing:** All new logic should be accompanied by tests.
- For more details, see `CONTRIBUTING.md`.
Showcase integration tests are run against a local server that implements the Showcase API. The [java-showcase/README.md](java-showcase/README.md) file provides detailed instructions on how to run these tests.
Loading