Rework CONTRIBUTING.md, extracting bits to new /docs/knowledge/* for … (#8303)

Co-authored-by: Trask Stalnaker <trask.stalnaker@gmail.com>

Author: jack-berg

AGENTS.md

@@ -1,8 +1,11 @@
 # AGENTS.md
 
-Read [CONTRIBUTING.md](CONTRIBUTING.md) first. It is the source of truth for repository layout,
-build and test commands, style expectations, and scope.
+Start with [docs/knowledge/README.md](docs/knowledge/README.md). It is the entry point for
+coding patterns, build conventions, testing guidance, and API stability rules. Load documents
+based on the scope of your work — the README contains a table mapping topics to load conditions.
 
-Additional guidance for agents:
+For project scope, PR process, and CLA requirements, see [CONTRIBUTING.md](CONTRIBUTING.md).
+
+## Additional guidance
 
 * Prefer small, localized changes over broad refactors.

CONTRIBUTING.md

@@ -0,0 +1,28 @@
+# Knowledge Index
+
+Repository guidance for coding and review, written to be useful for both humans and machines.
+
+**For humans**: these documents use plain language and examples rather than rigid rules. Where
+something is a strong convention, it's described as such. Where something is a matter of
+judgment, it's described that way too.
+
+**For machines**: load only files relevant to the current scope — the "Load when" column below
+is the signal.
+
+## Topics
+
+| File | Load when |
+| --- | --- |
+| [build.md](build.md) | Always — build requirements and common tasks |
+| [general-patterns.md](general-patterns.md) | Always — style, nullability, visibility, AutoValue, locking, logging |
+| [api-stability.md](api-stability.md) | Public API additions, removals, renames, or deprecations; stable vs alpha compatibility |
+| [gradle-conventions.md](gradle-conventions.md) | `build.gradle.kts` or `settings.gradle.kts` changes; new modules |
+| [testing-patterns.md](testing-patterns.md) | Test files in scope — assertions, test utilities, test suites |
+| [other-tasks.md](other-tasks.md) | Dev environment setup, benchmarks, composite builds, OTLP protobuf updates |
+
+## Conventions
+
+- File names are kebab-cased and topic-oriented. Most follow a `<domain>-<focus>.md` pattern
+  (e.g. `api-stability.md`, `testing-patterns.md`).
+- Sections within each document are ordered alphabetically, with the exception of any
+  introductory content placed directly under the document title.

docs/jmh.md

@@ -0,0 +1,73 @@
+# API Stability and Breaking Changes
+
+See [VERSIONING.md](../../VERSIONING.md) for the full versioning and compatibility policy.
+
+## Breaking changes in alpha modules
+
+Breaking changes are allowed in alpha modules but should still be approached carefully:
+
+- Prefer going through a deprecation cycle to ease migration, unless the deprecation would be too
+  cumbersome or the old API is genuinely unusable.
+- Bundle breaking changes together where possible to reduce churn for consumers.
+- Deprecations in alpha modules are typically introduced and removed within one release cycle
+  (approximately one month).
+
+## Breaking changes in stable modules
+
+Breaking changes are not allowed in stable modules outside of a major version bump. The build
+enforces this via japicmp — see [japicmp](#japicmp) below.
+
+## Deprecating API
+
+Applies to both stable and alpha modules. Use plain `@Deprecated` — do **not** use
+`forRemoval = true` or `since = "..."` (Java 8 compatibility requirement).
+
+```java
+/**
+ * @deprecated Use {@link #newMethod()} instead.
+ */
+@Deprecated
+public ReturnType oldMethod() {
+  return newMethod(); // delegate to replacement
+}
+```
+
+Rules:
+- Include a `@deprecated` Javadoc tag naming the replacement.
+- The deprecated method must delegate to its replacement, not the other way around — this ensures
+  overriders of the old method still get called.
+- Deprecated items in stable modules cannot be removed until the next major version.
+- Deprecated items in alpha modules should indicate when they will be removed (e.g.
+  `// to be removed in 1.X.0`). It is also common to log a warning when a deprecated-for-removal
+  feature is used, to increase visibility for consumers.
+- Add the `deprecation` label to the PR.
+
+## japicmp
+
+`otel.japicmp-conventions` runs the `jApiCmp` task as part of `check` for every published module.
+It compares the locally-built jar against the latest release to detect breaking changes, and writes
+a human-readable diff to `docs/apidiffs/current_vs_latest/<artifact>.txt`.
+
+- Breaking changes in stable modules fail the build.
+- AutoValue classes are exempt from the abstract-method-added check — the generated implementation
+  handles those automatically.
+- The diff files are committed to the repo. Include any changes to them in your PR.
+- Run `./gradlew jApiCmp` to regenerate diffs after API changes.
+
+## Stable vs alpha modules
+
+Artifacts without an `-alpha` version suffix are **stable**. Artifacts with `-alpha` have no
+compatibility guarantees and may break on every release. The `internal` package is always exempt
+from compatibility guarantees regardless of artifact stability.
+
+To mark a module as alpha, add a `gradle.properties` file at the module root containing:
+
+```properties
+otel.release=alpha
+```
+
+See [`exporters/prometheus/gradle.properties`](../../exporters/prometheus/gradle.properties) for
+an example.
+
+See [VERSIONING.md](../../VERSIONING.md) for the full compatibility policy, including what
+constitutes a source-incompatible vs binary-incompatible change.

docs/knowledge/README.md

@@ -0,0 +1,36 @@
+# Build
+
+Java 21 or higher is required to build. The built artifacts target Java 8 or higher.
+
+## Common tasks
+
+```bash
+# Fix formatting violations
+./gradlew spotlessApply
+
+# Run checks only (tests + static analysis + formatting verification)
+./gradlew check
+
+# Build, run tests, static analysis, and check formatting
+./gradlew build
+```
+
+All tasks can be scoped to a single module by prefixing with the module path:
+
+```bash
+./gradlew :sdk:metrics:spotlessApply
+./gradlew :sdk:metrics:check
+./gradlew :sdk:metrics:build
+```
+
+`./gradlew build` and `./gradlew check` both depend on the `jApiCmp` task, which compares the
+locally-built jars against the latest release and writes diffs to `docs/apidiffs/current_vs_latest/`.
+Include any changes to those files in your PR. See [api-stability.md](api-stability.md#japicmp)
+for details.
+
+If your branch is not up to date with `main`, `jApiCmp` may produce a diff that reflects changes
+already merged to `main` rather than your own changes. Rebase or merge `main` before treating
+the diff as meaningful.
+
+For dev environment setup, composite builds, OTLP protobuf updates, and other tasks, see
+[other-tasks.md](other-tasks.md).

docs/knowledge/api-stability.md

@@ -0,0 +1,135 @@
+# General Patterns
+
+## @Nullable
+
+All arguments and members are treated as non-null by default. Annotate with `@Nullable` (from
+`javax.annotation`) only when `null` is actually possible.
+
+- **Fields**: annotate only if the field can hold `null` after construction.
+- **Parameters**: annotate only if `null` is actually passed by callers.
+- **Return types**: annotate only if the method actually returns `null`. A non-null implementation
+  of a `@Nullable`-declared interface method should omit the annotation — it is more precise.
+
+## API consistency
+
+The project aims to provide a consistent experience across all public APIs. When designing new
+API, look for prior art in the project before introducing new patterns — prefer the style already
+established in the same package, then the same module, then the broader project.
+
+## AutoValue
+
+Use [AutoValue](https://github.com/google/auto/tree/master/value) for new value classes.
+
+- Add `annotationProcessor("com.google.auto.value:auto-value")` in `build.gradle.kts`.
+- `auto-value-annotations` is already available via `otel.java-conventions`.
+- Add a package-private constructor to all AutoValue classes to prevent external extension.
+- `japicmp` allows new abstract methods on AutoValue classes — the `AllowNewAbstractMethodOnAutovalueClasses` rule handles this.
+
+## Class member ordering
+
+In general, order class members as follows:
+
+1. Static fields (final before non-final)
+2. Instance fields (final before non-final)
+3. Constructors
+   - In static utility classes, the private constructor goes after methods, not before.
+4. Methods
+   - If methods call each other, the calling method should appear above the method it calls.
+5. Nested classes
+
+## Dedicated lock objects
+
+Do not synchronize using a class's intrinsic lock (`synchronized(this)` or `synchronized` method).
+Use a dedicated lock object:
+
+```java
+private final Object lock = new Object();
+
+public void doSomething() {
+  synchronized (lock) { ... }
+}
+```
+
+## Formatting
+
+Formatting is enforced by three tools. Run `./gradlew spotlessApply` before committing — it fixes
+most violations automatically.
+
+- **Spotless** — formatting rules vary by file type (see `buildSrc/src/main/kotlin/otel.spotless-conventions.gradle.kts`):
+  Java uses google-java-format; Kotlin uses ktlint. Also enforces the Apache license header
+  (template in `buildscripts/spotless.license.java`) and misc file hygiene (trailing whitespace,
+  final newline, etc.).
+- **Checkstyle** — enforces naming conventions, import ordering, Javadoc structure, and other
+  rules not covered by formatting alone. Config is in `buildscripts/checkstyle.xml`.
+- **EditorConfig** (`.editorconfig`) — configures IntelliJ to match project style automatically.
+  It doesn't cover all rules, so `spotlessApply` is still required.
+
+## Internal code
+
+Prefer package-private over putting code in an `internal` package. Use `internal` only when the
+code must be `public` for technical reasons (e.g. accessed across packages within the same module)
+but should not be part of the public API.
+
+Public classes in `internal` packages are excluded from semver guarantees and Javadoc, but they
+must carry one of the two standard disclaimers (enforced by the `OtelInternalJavadoc` Error Prone
+check in `custom-checks/`):
+
+```java
+// Standard internal disclaimer:
+/**
+ * This class is internal and is hence not for public use. Its APIs are unstable and can change
+ * at any time.
+ */
+
+// For incubating internal code that may be promoted to public API:
+/**
+ * This class is internal and experimental. Its APIs are unstable and can change at any time.
+ * Its APIs (or a version of them) may be promoted to the public stable API in the future, but
+ * no guarantees are made.
+ */
+```
+
+Internal code must not be used across module boundaries — module `foo` must not call internal
+code from module `bar`. Cross-module internal usage is a known issue being tracked and cleaned up
+in open-telemetry/opentelemetry-java#6970.
+
+## Javadoc
+
+All public classes, and their public and protected methods, must have complete Javadoc including
+all parameters — this is enforced for public APIs. Package-private and private members may have
+Javadoc at the author's discretion.
+
+- No `@author` tags.
+- New public API elements require a `@since` annotation. This is added automatically during the
+  [release process](../../RELEASING.md) — do not include it in your PR.
+- See [section 7.3.1](https://google.github.io/styleguide/javaguide.html#s7.3.1-javadoc-exception-self-explanatory)
+  for self-explanatory exceptions.
+
+Published Javadoc is available at https://javadoc.io/doc/io.opentelemetry.
+
+## Language version compatibility
+
+Production code targets Java 8. Test code also targets Java 8. Do not use Java 9+ APIs unless
+the module explicitly sets a higher minimum. See [VERSIONING.md](../../VERSIONING.md) for the full
+language version compatibility policy, including Android and Kotlin minimum versions.
+
+## Logging
+
+Use `java.util.logging` (JUL) in production source sets. Do not use SLF4J or other logging
+frameworks in `src/main/`. Tests bridge JUL to SLF4J via `JulBridgeInitializer` (configured
+automatically by `otel.java-conventions`).
+
+```java
+private static final Logger LOGGER = Logger.getLogger(MyClass.class.getName());
+```
+
+## toString()
+
+Adding `toString()` overrides is encouraged for debugging assistance. All `toString()`
+implementations should be considered unstable unless explicitly documented otherwise — do not
+rely on their output programmatically.
+
+## Visibility
+
+Use the most restrictive access modifier that still allows the code to function correctly. Use
+`final` on classes unless extension is explicitly intended.

docs/knowledge/build.md

@@ -0,0 +1,78 @@
+# Gradle Conventions
+
+## AutoValue in build files
+
+See [general-patterns.md](general-patterns.md#autovalue) for the AutoValue usage pattern.
+`auto-value-annotations` is provided globally by `otel.java-conventions`. Only add the
+annotation processor if the module uses AutoValue in production code:
+
+```kotlin
+dependencies {
+  annotationProcessor("com.google.auto.value:auto-value")        // production
+  testAnnotationProcessor("com.google.auto.value:auto-value")    // tests only
+}
+```
+
+## Convention plugins
+
+Every module applies a base set of convention plugins from `buildSrc/src/main/kotlin/`.
+
+| Plugin | Purpose | Who applies it |
+| --- | --- | --- |
+| `otel.java-conventions` | Base Java toolchain, Checkstyle, Spotless, Error Prone, test config | All modules |
+| `otel.publish-conventions` | Maven publishing, POM generation | Published (non-internal) modules |
+| `otel.animalsniffer-conventions` | Android API level compatibility checking | Modules targeting Android |
+| `otel.jmh-conventions` | JMH benchmark support | Modules with benchmarks |
+| `otel.japicmp-conventions` | API diff generation against latest release | Published modules (applied by `otel.publish-conventions`) |
+| `otel.protobuf-conventions` | Protobuf code generation | Protobuf modules only |
+
+A typical published module:
+
+```kotlin
+plugins {
+  id("otel.java-conventions")
+  id("otel.publish-conventions")
+}
+```
+
+A testing-internal module (shared test utilities, not published):
+
+```kotlin
+plugins {
+  id("otel.java-conventions")
+  // no otel.publish-conventions
+}
+```
+
+## Dependency resolution
+
+All external dependency versions are managed by `:dependencyManagement` (a BOM). Do not
+specify versions directly in `build.gradle.kts` — add new entries to the BOM if a version
+needs to be pinned.
+
+`otel.java-conventions` configures `failOnVersionConflict()` and `preferProjectModules()`
+globally. Do not override these without a strong reason.
+
+## Module naming
+
+Every module must declare its Java module name:
+
+```kotlin
+otelJava.moduleName.set("io.opentelemetry.sdk.metrics")
+```
+
+When the archive name cannot be derived correctly from the project name, set it explicitly:
+
+```kotlin
+base.archivesName.set("opentelemetry-api")
+```
+
+## `settings.gradle.kts` ordering
+
+New `include(...)` entries must be in **alphabetical order** within their surrounding group. The
+file is large — find the right lexicographic position before inserting.
+
+## Shared test utilities and test suites
+
+See [testing-patterns.md](testing-patterns.md) for shared test utility patterns and test suite
+registration.