Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/agents/knowledge/general-rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ When a "Knowledge File" is listed, load it from `knowledge/` before reviewing th
| Semconv | Dual semconv testing | `SemconvStability`, `maybeStable`, semconv Gradle tasks | `testing-semconv-stability.md` |
| Testing | General test patterns | Test files in scope — assertion style, test method signatures and throws clauses, resource cleanup, attribute assertions | `testing-general-patterns.md` |
| Testing | Experimental flag tests | `testExperimental`, experimental attribute assertions, `experimental` flags in JVM args or system properties | `testing-experimental-flags.md` |
| Testing | Flag-gated / mode-dependent assertion shape (experimental, `testLatestDeps`, semconv) — shared accessor (`testLatestDeps()`, `emitStable*Semconv()`) or `EXPERIMENTAL_ATTRIBUTES` constant, inline ternary with `null` for "absent" | Test classes branching on `EXPERIMENTAL_ATTRIBUTES`, `testLatestDeps()`, or `emitOld*`/`emitStable*` | `testing-general-patterns.md` |
| Library | TelemetryBuilder/getter/setter patterns | Library instrumentation classes | `library-patterns.md` |
| API | Deprecation and breaking-change policy | Public API changes | `api-deprecation-policy.md` |
| Config | Config property stability/renames/removals | `otel.instrumentation.*` property changes, `DeclarativeConfigUtil` or `ConfigProperties` usage | `config-property-stability.md` |
Expand Down
32 changes: 20 additions & 12 deletions .github/agents/knowledge/testing-experimental-flags.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,9 @@ block but there is no dedicated `testExperimental` task, fix it:

1. Create a `testExperimental` task (see Gradle Task Setup below).
2. Move the experimental flag out of the shared/default task config into `testExperimental`.
3. Add a `private static final boolean EXPERIMENTAL_ATTRIBUTES` field to test classes.
4. Wrap experimental attribute assertions with the `experimental()` helper.
5. Wire the new task into `check`.
3. Add a `private static final boolean EXPERIMENTAL_ATTRIBUTES` field to test classes and
gate experimental attribute assertions on it (see Java Test Patterns below).
4. Wire the new task into `check`.

## Gradle Task Setup

Expand All @@ -51,30 +51,38 @@ val testExperimental by registering(Test::class) {

## Java Test Patterns

Read the flag once into a `private static final boolean` at class level:
For the cross-cutting shape — inline ternary with `null` for "absent", when to use top-level
`if` blocks, and `assumeTrue(...)` guidance — see
[testing-general-patterns.md](testing-general-patterns.md#flag-gated--mode-dependent-assertions).
The experimental-specific patterns below build on that shape.

### Hoist the flag into a per-class `EXPERIMENTAL_ATTRIBUTES` constant

The property name is module-specific, so there is no shared accessor. Read it once into a
`private static final boolean` near the top of the class and reference the constant
everywhere in the file:

```java
private static final boolean EXPERIMENTAL_ATTRIBUTES =
Boolean.getBoolean("otel.instrumentation.<module>.experimental-span-attributes");
```

Use inline ternary in assertions — `null` means attribute expected absent:
When multiple experimental flags coexist in one class, use a more specific name per flag.

```java
equalTo(ExperimentalAttributes.SOME_ATTR, EXPERIMENTAL_ATTRIBUTES ? "value" : null)
```
### Single-arg `experimental(value)` helper

When many assertions share the flag, extract an `experimental()` helper:
Experimental attributes are by definition absent when the flag is off, so the off-branch is
always `null`. When several assertions in the same class gate attributes on
`EXPERIMENTAL_ATTRIBUTES`, extract a tiny helper:

```java
@Nullable
private static <T> T experimental(T value) {
return EXPERIMENTAL_ATTRIBUTES ? value : null;
}

equalTo(SOME_KEY, experimental("value"))
```

For multiple test classes sharing the same flag, move the helper into a shared
`ExperimentalTestHelper` class and static-import it.

Use `assumeTrue(EXPERIMENTAL_ATTRIBUTES)` only when an entire test is meaningful in
experimental mode only — prefer the ternary/helper pattern so both modes are exercised.
38 changes: 38 additions & 0 deletions .github/agents/knowledge/testing-general-patterns.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,3 +135,41 @@ unless the span ordering within a trace is genuinely non-deterministic (e.g., co
producers/consumers, thread-pool fan-out, or channel interleaving). Sequential operations
like `repeat {}` loops, single-child traces, and `flux` sequential emission produce spans
in deterministic order — use `hasSpansSatisfyingExactly` for those.

## Flag-Gated / Mode-Dependent Assertions

Several test modes change which attributes, span names, status codes, or span indexes are
expected:

- Experimental attributes (`-Dotel.instrumentation.<module>.experimental-*=true`) — see
[testing-experimental-flags.md](testing-experimental-flags.md).
- Semconv stability (`-Dotel.semconv-stability.opt-in=...`) — see
[testing-semconv-stability.md](testing-semconv-stability.md).
- `testLatestDeps` Gradle property — runs against the newest supported library versions
instead of the pinned earliest-supported ones.

### Read the flag through a shared static helper, not a per-class field

Each flag has a shared static accessor; static-import it and call it directly. Never call
`Boolean.getBoolean("…")` inline and never duplicate the property-name string at the call
site.

| Flag | Shared accessor | Where it lives |
| --- | --- | --- |
| `-PtestLatestDeps=true` | `testLatestDeps()` | `io.opentelemetry.instrumentation.testing.util.TestLatestDeps` (testing-common) |
| `otel.semconv-stability.opt-in=…` | `emitStableDatabaseSemconv()`, `emitOldDatabaseSemconv()`, `emitStableCodeSemconv()`, etc. | `io.opentelemetry.instrumentation.api.internal.SemconvStability` |
| `otel.instrumentation.<module>.experimental-*` | per-module `EXPERIMENTAL_ATTRIBUTES` constant — see [testing-experimental-flags.md](testing-experimental-flags.md) | within the test class |

### Inline ternary in `equalTo(...)` with `null` for "absent"

Push the ternary as deep as possible — into the `equalTo` value or single attribute key —
rather than duplicating two whole `hasAttributesSatisfyingExactly(...)` blocks under a
`flag ? a : b`. The assertion API treats `null` as "expect attribute absent":

```java
equalTo(DB_USER, emitStableDatabaseSemconv() ? null : USER_DB)
equalTo(ERROR_TYPE, emitStableDatabaseSemconv() ? "42601" : null)
equalTo(SOME_KEY, EXPERIMENTAL_ATTRIBUTES ? "value" : null)
span.hasName(testLatestDeps() ? "GET" : "HTTP GET")
.hasParent(trace.getSpan(testLatestDeps() ? 0 : 1))
```
33 changes: 17 additions & 16 deletions .github/agents/knowledge/testing-semconv-stability.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,24 +76,30 @@ for other domains `check { dependsOn(testStableSemconv) }`.

## Asserting Attributes in Tests

Preferred approach (most compact) — inline ternary with `emitStable*()`, where `null` means
the attribute is expected absent:
For the cross-cutting shape — inline ternary with `null` for "absent", static-imported flag
accessors, and `assumeTrue(...)` guidance — see
[testing-general-patterns.md](testing-general-patterns.md#flag-gated--mode-dependent-assertions).
The semconv-specific patterns below build on that shape.

```java
equalTo(DB_USER, emitStableDatabaseSemconv() ? null : USER_DB)
equalTo(ERROR_TYPE, emitStableDatabaseSemconv() ? "42601" : null)
span.hasName(emitStableDatabaseSemconv() ? "SELECT" : "SELECT dbname")
```
### `maybeStable(OLD_KEY)` for 1:1 key renames

Use `maybeStable(oldKey)` when only the attribute key changes and the value stays the same.
`maybeStable()` does NOT cover `dup` mode — use separate `if` blocks for that.
Use `maybeStable(OLD_KEY)` when only the attribute *key* flips between old and stable
semconv and the value is identical:

```java
span.hasAttribute(equalTo(maybeStable(DB_STATEMENT), "SELECT ?"));
```

Use separate `if` blocks (not `if/else`) when assertion structure differs significantly between
modes — this ensures both branches execute in `dup` mode:
`maybeStable()` does **not** cover `/dup` mode (it returns one key, not both), and does
**not** apply where the mapping isn't 1:1 — for example `DB_RESPONSE_STATUS_CODE` →
`ERROR_TYPE`. Use `emitOld*()` / `emitStable*()` `if` blocks for those.

### `if` blocks (not `if/else`) when structure differs

When the *set* of asserted attributes differs between modes — not just values — use
separate top-level `if` blocks rather than `if/else`. For domains that support `/dup` mode
(currently RPC), this is required so both branches run; for other domains it's a habit
that keeps the assertion `/dup`-safe if the domain ever adopts it:

```java
if (emitStableCodeSemconv()) {
Expand All @@ -104,11 +110,6 @@ if (emitOldCodeSemconv()) {
}
```

Use `assumeTrue(emitStable*())` only when an entire test is meaningful in one mode only.

## Key Rules

- Add `@SuppressWarnings("deprecation")` at class level when tests use old Semconv constants.
- Use `if` (not `if/else`) for dual-mode assertions so both branches run in `/dup` mode.
- Do NOT use `maybeStable()` for `DB_RESPONSE_STATUS_CODE` → `ERROR_TYPE` — these don't have
a 1:1 mapping. Use `emitOld*()`/`emitStable*()` `if` blocks instead.
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,11 @@ class StaticImportFormatter : FormatterFunc.NeedsFile, Serializable {
"io.opentelemetry.instrumentation.api.internal.SemconvStability",
"emit[a-zA-Z0-9]*",
),
Rule(
"TestLatestDeps",
"io.opentelemetry.instrumentation.testing.util.TestLatestDeps",
"testLatestDeps",
),
Rule(
"SqlDialect",
"io.opentelemetry.instrumentation.api.incubator.semconv.db.SqlDialect",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@

package io.opentelemetry.instrumentation.apachedubbo.v2_7;

import static io.opentelemetry.instrumentation.testing.util.TestLatestDeps.testLatestDeps;
import static java.util.Collections.singletonMap;
import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.Mockito.when;
Expand Down Expand Up @@ -32,7 +33,7 @@ void testKeys() throws ReflectiveOperationException {
when(context.getLocalAddress()).thenReturn(new InetSocketAddress(1));

// for latest dep tests call getObjectAttachments, otherwise call getAttachments
if (Boolean.getBoolean("testLatestDeps")) {
if (testLatestDeps()) {
when(getObjectAttachments()).thenReturn(singletonMap("key", "value"));
} else {
when(rpcInvocation.getAttachments()).thenReturn(singletonMap("key", "value"));
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
import static io.opentelemetry.instrumentation.api.internal.SemconvStability.emitStableRpcSemconv;
import static io.opentelemetry.instrumentation.testing.GlobalTraceUtil.runWithSpan;
import static io.opentelemetry.instrumentation.testing.junit.service.SemconvServiceStabilityUtil.maybeStablePeerService;
import static io.opentelemetry.instrumentation.testing.util.TestLatestDeps.testLatestDeps;
import static io.opentelemetry.sdk.testing.assertj.OpenTelemetryAssertions.assertThat;
import static io.opentelemetry.sdk.testing.assertj.OpenTelemetryAssertions.equalTo;
import static io.opentelemetry.sdk.testing.assertj.OpenTelemetryAssertions.satisfies;
Expand Down Expand Up @@ -186,7 +187,7 @@ void testApacheDubboBase() throws ReflectiveOperationException {
: "hello"),
equalTo(
maybeStablePeerService(),
hasServicePeerName() && Boolean.getBoolean("testLatestDeps")
hasServicePeerName() && testLatestDeps()
? "test-peer-service"
: null),
satisfies(
Expand Down Expand Up @@ -370,7 +371,7 @@ void testApacheDubboTest() throws ReflectiveOperationException {
: "hello"),
equalTo(
maybeStablePeerService(),
hasServicePeerName() && Boolean.getBoolean("testLatestDeps")
hasServicePeerName() && testLatestDeps()
? "test-peer-service"
: null),
satisfies(
Expand Down Expand Up @@ -484,7 +485,7 @@ static void assertNetworkType(AbstractStringAssert<?> stringAssert) {

static void assertLatestDeps(
AbstractAssert<?, ?> assertion, Consumer<AbstractAssert<?, ?>> action) {
if (Boolean.getBoolean("testLatestDeps")) {
if (testLatestDeps()) {
action.accept(assertion);
} else {
assertion.isNull();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
import static io.opentelemetry.instrumentation.api.internal.SemconvStability.emitStableRpcSemconv;
import static io.opentelemetry.instrumentation.testing.GlobalTraceUtil.runWithSpan;
import static io.opentelemetry.instrumentation.testing.junit.service.SemconvServiceStabilityUtil.maybeStablePeerService;
import static io.opentelemetry.instrumentation.testing.util.TestLatestDeps.testLatestDeps;
import static io.opentelemetry.sdk.testing.assertj.OpenTelemetryAssertions.assertThat;
import static io.opentelemetry.sdk.testing.assertj.OpenTelemetryAssertions.equalTo;
import static io.opentelemetry.sdk.testing.assertj.OpenTelemetryAssertions.satisfies;
Expand Down Expand Up @@ -229,7 +230,7 @@ void testDubboChain() throws ReflectiveOperationException {
: "hello"),
equalTo(
maybeStablePeerService(),
hasServicePeerName() && Boolean.getBoolean("testLatestDeps")
hasServicePeerName() && testLatestDeps()
? "test-peer-service"
: null),
satisfies(
Expand Down Expand Up @@ -285,7 +286,7 @@ void testDubboChain() throws ReflectiveOperationException {
: "hello"),
equalTo(
maybeStablePeerService(),
hasServicePeerName() && Boolean.getBoolean("testLatestDeps")
hasServicePeerName() && testLatestDeps()
? "test-peer-service"
: null),
satisfies(
Expand Down Expand Up @@ -510,7 +511,7 @@ void testDubboChainInJvm() throws ReflectiveOperationException {
: "hello"),
equalTo(
maybeStablePeerService(),
hasServicePeerName() && Boolean.getBoolean("testLatestDeps")
hasServicePeerName() && testLatestDeps()
? "test-peer-service"
: null),
satisfies(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@

import static io.opentelemetry.instrumentation.testing.junit.http.AbstractHttpClientTest.CONNECTION_TIMEOUT;
import static io.opentelemetry.instrumentation.testing.junit.http.AbstractHttpClientTest.READ_TIMEOUT;
import static io.opentelemetry.instrumentation.testing.util.TestLatestDeps.testLatestDeps;

import io.opentelemetry.instrumentation.testing.junit.InstrumentationExtension;
import io.opentelemetry.instrumentation.testing.junit.http.HttpClientInstrumentationExtension;
Expand Down Expand Up @@ -224,7 +225,7 @@ protected void configure(HttpClientTestOptions.Builder optionsBuilder) {
// apparently apache http client does not report the 302 status code?
optionsBuilder.setResponseCodeOnRedirectError(null);

if (Boolean.getBoolean("testLatestDeps")) {
if (testLatestDeps()) {
optionsBuilder.disableTestHttps();
optionsBuilder.disableTestRemoteConnection();
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@

package io.opentelemetry.javaagent.instrumentation.armeria.v1_3;

import static io.opentelemetry.instrumentation.testing.util.TestLatestDeps.testLatestDeps;

import com.linecorp.armeria.server.ServerBuilder;
import io.opentelemetry.instrumentation.armeria.v1_3.AbstractArmeriaHttpServerTest;
import io.opentelemetry.instrumentation.testing.junit.InstrumentationExtension;
Expand All @@ -31,7 +33,7 @@ protected void configure(HttpServerTestOptions options) {
options.setTestHttpPipelining(false);
// span for non-standard request is created by netty instrumentation when not running latest
// dep tests
if (Boolean.getBoolean("testLatestDeps")) {
if (testLatestDeps()) {
options.disableTestNonStandardHttpMethod();
} else {
options.setResponseCodeOnNonStandardHttpMethod(405);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@

package io.opentelemetry.javaagent.instrumentation.asynchttpclient.v1_8;

import static io.opentelemetry.instrumentation.testing.util.TestLatestDeps.testLatestDeps;
import static io.opentelemetry.semconv.NetworkAttributes.NETWORK_PROTOCOL_VERSION;

import com.ning.http.client.AsyncCompletionHandler;
Expand Down Expand Up @@ -104,7 +105,7 @@ protected void configure(HttpClientTestOptions.Builder optionsBuilder) {
optionsBuilder.disableTestHttps();

// disable read timeout test for non latest because it is flaky with 1.8.x
if (!Boolean.getBoolean("testLatestDeps")) {
if (!testLatestDeps()) {
optionsBuilder.disableTestReadTimeout();
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@

package io.opentelemetry.javaagent.instrumentation.asynchttpclient.v1_9;

import static io.opentelemetry.instrumentation.testing.util.TestLatestDeps.testLatestDeps;
import static io.opentelemetry.semconv.NetworkAttributes.NETWORK_PROTOCOL_VERSION;

import com.ning.http.client.AsyncCompletionHandler;
Expand Down Expand Up @@ -99,7 +100,7 @@ protected void configure(HttpClientTestOptions.Builder optionsBuilder) {
optionsBuilder.disableTestRedirects();

// disable read timeout test for non latest because it is flaky with 1.9.0
if (!Boolean.getBoolean("testLatestDeps")) {
if (!testLatestDeps()) {
optionsBuilder.disableTestReadTimeout();
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@

package io.opentelemetry.javaagent.instrumentation.asynchttpclient.v2_0;

import static io.opentelemetry.instrumentation.testing.util.TestLatestDeps.testLatestDeps;

import io.opentelemetry.instrumentation.testing.junit.InstrumentationExtension;
import io.opentelemetry.instrumentation.testing.junit.http.AbstractHttpClientTest;
import io.opentelemetry.instrumentation.testing.junit.http.HttpClientInstrumentationExtension;
Expand Down Expand Up @@ -46,11 +48,10 @@ private static DefaultAsyncHttpClientConfig.Builder configureTimeout(

private static void setTimeout(
DefaultAsyncHttpClientConfig.Builder builder, String methodName, int timeout) {
boolean testLatestDeps = Boolean.getBoolean("testLatestDeps");
try {
Method method =
builder.getClass().getMethod(methodName, testLatestDeps ? Duration.class : int.class);
method.invoke(builder, testLatestDeps ? Duration.ofMillis(timeout) : timeout);
builder.getClass().getMethod(methodName, testLatestDeps() ? Duration.class : int.class);
method.invoke(builder, testLatestDeps() ? Duration.ofMillis(timeout) : timeout);
} catch (Exception e) {
throw new IllegalStateException("Failed to set timeout " + methodName, e);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@

package io.opentelemetry.javaagent.instrumentation.awssdk.v2_2;

import static io.opentelemetry.instrumentation.testing.util.TestLatestDeps.testLatestDeps;

import io.opentelemetry.instrumentation.awssdk.v2_2.AbstractAws2LambdaTest;
import io.opentelemetry.instrumentation.testing.junit.AgentInstrumentationExtension;
import io.opentelemetry.instrumentation.testing.junit.InstrumentationExtension;
Expand All @@ -25,7 +27,7 @@ protected InstrumentationExtension getTesting() {
@Override
protected boolean canTestLambdaInvoke() {
// only supported since 2.17.0
return Boolean.getBoolean("testLatestDeps");
return testLatestDeps();
}

@Override
Expand Down
Loading
Loading