refactor(core,instr,agent,client): remove permission API and update instrumentation/tests

Removes legacy permission annotations and extension permission markers.

Adjusts core extension metadata, messages, and dtrace integration.

Updates instrumentation pipeline and verifiers; refreshes related tests and test data.

Touches client/agent entrypoints and messaging; polishes samples and README.

Build tweaks across modules and CI workflow.

Author: jbachorik

.github/workflows/continuous.yml

@@ -131,7 +131,7 @@ jobs:
       - name: Run tests
         run: |
           set +x
-          ./gradlew --no-daemon --parallel --build-cache  -Pintegration :integration-tests:test
+          ./gradlew --no-daemon --parallel --build-cache -Pintegration -PCI :integration-tests:test
       - name: Check Gradle cache size
         run: du -sh ~/.gradle/caches
       - name: Integration test reports

CONTRIBUTING.md

@@ -0,0 +1,84 @@
+# Contributing to BTrace
+
+Thanks for your interest in contributing! This guide covers local development, running tests, Gradle tips, and common troubleshooting.
+
+Note: Pull requests can only be accepted from signers of the Oracle Contributor Agreement (OCA). See the project README for details.
+
+## Local Development
+
+- JDK: Use a reasonably recent JDK (11+ recommended). The project targets a broad range but tests run comfortably on 11/17.
+- Wrapper: Use the bundled `./gradlew` wrapper. It will download the pinned Gradle version if needed.
+- Local Gradle cache (optional but recommended):
+  - macOS/Linux: `export GRADLE_USER_HOME="$PWD/.gradle-home"`
+  - Windows (PowerShell): `$env:GRADLE_USER_HOME = "$PWD/.gradle-home"`
+
+## Running Tests
+
+- All unit tests (skip integration tests):
+  ```sh
+  ./gradlew --no-daemon test -x integration-tests:test
+  ```
+
+- Per-module tests:
+  - Runtime: `./gradlew :btrace-runtime:test`
+  - Extension: `./gradlew :btrace-extension:test`
+  - Compiler: `./gradlew :btrace-compiler:test`
+  - Instr: `./gradlew :btrace-instr:test`
+
+- Update instrumentor goldens when bytecode output changes:
+  ```sh
+  ./gradlew test -PupdateTestData
+  ```
+
+- Integration tests (spawn JVMs, exercise agent and extensions):
+  ```sh
+  ./gradlew --no-daemon integration-tests:test
+  ```
+  - If tests fail due to denied privileged extensions, pass a policy file to the tested JVMs:
+    - Create `permissions.properties`:
+      ```properties
+      allowPrivileged=true
+      allowExtensions=btrace-metrics,btrace-utils
+      ```
+    - Export path: `export BTRACE_PERMS=$PWD/permissions.properties`
+    - Run Gradle with: `-Dbtrace.permissions=$BTRACE_PERMS`
+
+## Gradle Tips
+
+- Prefer IPv4 if your environment has unusual local IP settings (helps Gradle select a wildcard address):
+  ```sh
+  export GRADLE_OPTS="-Djava.net.preferIPv4Stack=true -Djava.net.preferIPv6Addresses=false"
+  ```
+- Enable Gradle debug output for flakiness: add `--info` or `--debug`.
+- Run a single test class/method:
+  ```sh
+  ./gradlew :btrace-extension:test --tests org.openjdk.btrace.extension.ExtensionBridgeImplPolicyTest
+  ./gradlew :btrace-runtime:test --tests "*ExtensionIndyShimIndexTest.resolvesNoopShimFromIndex"
+  ```
+
+## Troubleshooting
+
+- Gradle wrapper needs to download Gradle: ensure network is allowed once; subsequent runs use the local cache under `.gradle-home`.
+- Error: `Could not determine a usable wildcard IP for this machine`:
+  - Set the IPv4 flags shown above or ensure local networking is available.
+- Permission errors when Gradle writes outside the workspace:
+  - Use a local Gradle cache via `GRADLE_USER_HOME` as shown above.
+- Integration tests failing with permissions denied:
+  - Provide a policy file and pass it via `-Dbtrace.permissions=/path/to/permissions.properties`.
+
+## Code Style & Scope
+
+- Keep changes focused and minimal; align with existing code style.
+- Update docs when changing user-visible behavior.
+- Prefer clear separation of concerns and small helpers over inlined, complex logic.
+- Avoid introducing new dependencies without discussion.
+
+## Submitting a PR
+
+1. Fork the repo and branch from `develop` (unless otherwise agreed).
+2. Make your changes and run tests locally.
+3. If instrumentor behavior changed, update goldens (`-PupdateTestData`) and include them in your commit.
+4. Submit a PR with a concise description of the change, rationale, and any follow-ups.
+
+Happy tracing!
+

README.md

@@ -42,6 +42,30 @@ When instrumentor code changes, update test golden files with:
 ```
 Commit the regenerated golden files to Git.
 
+### How To Run Tests
+
+For a fast local cycle, run unit tests and skip integration tests (which start forked JVMs):
+
+```sh
+export GRADLE_USER_HOME="$PWD/.gradle-home"   # keep caches inside the repo (optional)
+./gradlew --no-daemon test -x integration-tests:test
+```
+
+Tips:
+- Prefer IPv4 if your environment has odd local IPs: set `GRADLE_OPTS="-Djava.net.preferIPv4Stack=true -Djava.net.preferIPv6Addresses=false"`.
+- Run specific modules:
+  - Runtime: `./gradlew :btrace-runtime:test`
+  - Extension: `./gradlew :btrace-extension:test`
+  - Compiler: `./gradlew :btrace-compiler:test`
+  - Instr: `./gradlew :btrace-instr:test`
+- Update instrumentor golden files when bytecode output changes: `./gradlew test -PupdateTestData`.
+
+Integration tests (optional):
+```sh
+./gradlew --no-daemon integration-tests:test
+```
+These may exercise privileged extensions. If you run into permission denials, provide a policy file and pass it to the test JVMs via `-Dbtrace.permissions=/path/to/permissions.properties`.
+
 
 ## Using BTrace
 
@@ -188,12 +212,7 @@ BTrace supports extensions (like StatsdExtension) that provide additional functi
 * **Standard permissions** (granted unless denied): FILE_READ, SYSTEM_PROPS, THREAD_INFO, MEMORY_INFO
 * **Privileged permissions** (require explicit grant): FILE_WRITE, NETWORK, THREADS, NATIVE, EXEC, REFLECTION, CLASSLOADER, UNLIMITED_MEMORY
 
-Declare required permissions in your probe with `@RequestPermission`:
-```java
-@BTrace
-@RequestPermission(Permission.NETWORK)
-public class MyProbe { ... }
-```
+Permissions are enforced based on extension/service descriptors and agent grants specified at attach-time.
 
 Grant permissions at runtime:
 ```sh
@@ -207,6 +226,47 @@ btrace -le <PID>
 
 See the [Tutorial](docs/BTraceTutorial.md) for detailed documentation.
 
+Extensions CLI: use `btracex` to inspect and manage extensions and the simplified permission policy:
+- `btracex inspect <zip|dir>` prints extension id, version, services, and whether it’s privileged.
+- `btracex policy print|set [--policy-file <path>|--home|--classpath <outDir>]` edits `allowExtensions`, `denyExtensions`, `allowPrivileged`.
+- `btracex list` shows installed extensions; `btracex install` installs from Maven coordinates.
+
+Note: Extension “required permissions” are informational and help operators assess risk. Implementation linking is controlled by per‑extension allow/deny lists and the `allowPrivileged` flag; when blocked, APIs remain available and SHIMs are used so probes continue safely.
+
+#### Agent Policy and Allow/Deny Lists
+- Launch-time policy can be set via agent args (operator-controlled):
+  - `-javaagent:btrace-agent.jar=...,grant=NETWORK,THREADS,grantAll=false`
+  - `-javaagent:btrace-agent.jar=...,allowExtensions=btrace-statsd,my-metrics,denyExtensions=legacy-foo`
+- Optional policy file (process-local): `-Dbtrace.permissions=/path/to/permissions.properties` or `~/.btrace/permissions.properties`.
+- When an extension impl is blocked, the API remains on bootstrap so SHIMs can be generated.
+
+See docs/PermissionPolicy.md for details and examples.
+
+#### btracex TUI (interactive)
+- Launch: `btracex inspect` (with no args) opens an interactive view of installed extensions.
+- Header: shows current policy file path and the list of scanned repositories.
+- Table: columns State, Id, Version. State uses compact symbols: `?` (default), `+` (allowed), `-` (denied).
+- Details: selection updates automatically; shows the full-word state: `default` / `allowed` / `denied` and the full path.
+- Legend: a short legend under the table maps the state symbols.
+
+Screenshot / demo (optional):
+
+![btracex TUI](docs/images/btracex-tui.png)
+
+![btracex TUI demo](docs/images/btracex-tui.gif)
+
+See also: docs/TUI.md for recording tips and an ASCII preview.
+
+Keys
+- Navigate: Arrow keys, PageUp/PageDown, Home/End
+- Toggle state: space (flows `? → + (confirm) → - → +`; only `c` clears to default)
+- Clear: `c` (removes extension id from both allow and deny lists)
+- Explain privileges: `e` (opens a dialog with required permissions and risk descriptions)
+- Filter: `/` (filter by id or path)
+- Sort: `s` (choose column; repeat to toggle asc/desc)
+- Adjust split: `m` (enter mode), then Up/Down to resize; press `Esc` or `m` again to exit
+- Help / Quit: `?` / `q`
+
 ### Maven Integration
 
 The [BTrace Maven Plugin](https://github.com/btraceio/btrace-maven) enables:

benchmarks/runtime-benchmarks/build.gradle

@@ -17,7 +17,7 @@ dependencies {
   implementation project(":btrace-compiler")
   jmh project(":btrace-instr")
   jmh project(":btrace-runtime")
-  jmh project(":btrace-statsd")
+  jmh project(":btrace-extensions:btrace-statsd")
   jmh libs.jmh
   jmh libs.jmh.annprocess
   compilerDeps project(path: ":btrace-dist", configuration: "shadow")
@@ -52,7 +52,7 @@ jmhJar {
   include "org/openjdk/btrace/instr/**"
   include 'org/openjdk/btrace/generated/**/*'
   include 'org/openjdk/btrace/runtime/**'
-  include 'org/openjdk/btrace/services/**'
+  // no legacy services classes to package
   include "joptsimple/**"
   include "org/apache/**"
   include '*.btclass'
@@ -66,4 +66,3 @@ jmh {
   includes = ['org.openjdk.btrace.bench.ClassFilterBenchmark']
   verbosity = 'EXTRA'
 }
-

btrace-agent/build.gradle

@@ -21,7 +21,12 @@ dependencies {
         implementation files("${toolsJar}")
     }
     implementation project(':btrace-core')
-    implementation project(':btrace-services-api')
     implementation project(':btrace-runtime')
     implementation project(':btrace-instr')
-}
+    implementation project(':btrace-extension')
+}
+
+// Exclude sources that rely on JDK-internal modules from Javadoc to avoid missing-package errors
+tasks.named('javadoc').configure {
+    exclude 'org/openjdk/btrace/agent/PerfReaderImpl.java'
+}

btrace-agent/src/main/java/org/openjdk/btrace/agent/Client.java

@@ -41,6 +41,9 @@
 import org.openjdk.btrace.core.comm.StatusCommand;
 import org.openjdk.btrace.core.extensions.Permission;
 import org.openjdk.btrace.core.extensions.PermissionSet;
+import org.openjdk.btrace.extension.ExtensionDescriptorDTO;
+import org.openjdk.btrace.extension.ExtensionLoader;
+import org.openjdk.btrace.extension.ExtensionRegistry;
 import org.openjdk.btrace.instr.BTraceProbe;
 import org.openjdk.btrace.instr.BTraceProbeFactory;
 import org.openjdk.btrace.instr.BTraceProbePersisted;
@@ -54,7 +57,6 @@
 import org.openjdk.btrace.instr.MethodTrackingContext;
 import org.openjdk.btrace.runtime.BTraceRuntimeAccess;
 import org.openjdk.btrace.runtime.BTraceRuntimes;
-import org.openjdk.btrace.runtime.ExtensionRegistry;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -348,6 +350,9 @@ final Class<?> loadClass(InstrumentCommand instr) throws IOException {
           throw new SecurityException(formatPermissionError(missing));
         }
       }
+
+      // Validate that all injected service types are declared by available extensions
+      validateDeclaredServices(probe);
     } catch (Throwable th) {
       log.debug("Failed to load BTrace probe code", th);
       errorExit(th);
@@ -393,6 +398,26 @@ final Class<?> loadClass(InstrumentCommand instr) throws IOException {
       sendCommand(new MessageCommand(warning.toString()));
     }
 
+    // Expose extension-declared permissions for integration visibility
+    // Print extension permissions only when explicitly requested (debug or system property)
+    if (settings.isDebug() || Boolean.getBoolean("btrace.list.extension.permissions")) {
+      try {
+        ExtensionLoader loader = Main.getExtensionLoader();
+        if (loader != null) {
+          StringBuilder info = new StringBuilder();
+          info.append("[BTRACE INFO] Extensions and declared permissions:\n");
+          for (ExtensionDescriptorDTO ext : loader.getAvailableExtensions()) {
+            PermissionSet perms = ext.getRequiredPermissions();
+            String pStr = perms != null && !perms.isEmpty() ? perms.toString() : "[]";
+            info.append("  - ").append(ext.getId()).append(": ").append(pStr).append("\n");
+          }
+          sendCommand(new MessageCommand(info.toString()));
+        }
+      } catch (Throwable t) {
+        // ignore, informational only
+      }
+    }
+
     boolean entered = false;
     try {
       entered = BTraceRuntimeAccess.enter(runtime);
@@ -408,6 +433,62 @@ final Class<?> loadClass(InstrumentCommand instr) throws IOException {
     }
   }
 
+  /**
+   * Validates that all {@code @Injected} service field types used by the given probe are
+   * declared by some available extension. This runs in the agent's runtime where the actual
+   * classloader and JPMS module layer apply.
+   *
+   * Why reflection here (vs. pure ASM):
+   * - Classloader identity: Ensures types are checked against the agent's classes loaded by the
+   *   correct loader. Name-only checks in ASM cannot detect split-brain issues (same FQN, different
+   *   loader/JAR) that would later cause ClassCastException.
+   * - JPMS access rules: Surfaces missing exports/opens and other module access constraints that
+   *   cannot be proven by static bytecode analysis.
+   * - Linkage/loadability: Fails fast if a referenced type is not actually resolvable on the
+   *   agent's runtime path (NoClassDefFoundError/missing transitive dependencies).
+   * - Assignability truth: Verifies that the service type corresponds to something an extension
+   *   actually declares in its manifest, avoiding false positives from shaded or version-skewed
+   *   classes.
+   *
+   * Implementation notes:
+   * - We use reflection only to access the probe's internal service field map (to avoid a direct
+   *   compile-time dependency on the probe's delegate type) and to keep the agent/probe boundary
+   *   clean. We do not instantiate user classes or trigger class initializers.
+   * - This check complements compile-time and bytecode-time validation (ASM-based) which enforce
+   *   structural rules without loading classes. Reflection here provides the necessary runtime
+   *   assurance in the actual environment where the agent will operate.
+   */
+  private void validateDeclaredServices(BTraceProbe probe) throws IOException {
+    if (!(probe instanceof BTraceProbePersisted)) {
+      return;
+    }
+    ExtensionLoader loader = Main.getExtensionLoader();
+    if (loader == null) {
+      return;
+    }
+    // Reflectively access serviceFields() from the delegate to get injected service types
+    try {
+      java.lang.reflect.Field delF = BTraceProbePersisted.class.getDeclaredField("delegate");
+      delF.setAccessible(true);
+      Object delegate = delF.get(probe);
+      java.lang.reflect.Method svcM = delegate.getClass().getDeclaredMethod("serviceFields");
+      svcM.setAccessible(true);
+      @SuppressWarnings("unchecked")
+      Map<String, String> svcMap = (Map<String, String>) svcM.invoke(delegate);
+      if (svcMap != null) {
+        for (String internalName : svcMap.values()) {
+          String fqcn = internalName.replace('/', '.');
+          if (loader.findExtensionForService(fqcn) == null) {
+            throw new IOException(
+                "Injected service type not declared by any extension: " + fqcn);
+          }
+        }
+      }
+    } catch (ReflectiveOperationException e) {
+      log.debug("Unable to inspect injected services for validation", e);
+    }
+  }
+
   protected void closeAll() throws IOException {
     if (flusher != null) {
       flusher.cancel();
@@ -521,7 +602,8 @@ boolean retransformLoaded() throws UnmodifiableClassException {
               log.debug("Attempting to retransform class: {}", c.getName());
               inst.retransformClasses(c);
             } catch (ClassFormatError | VerifyError e) {
-              log.debug("Class '{}' verification failed", c.getName(), e);
+              // Avoid printing full stack traces in debug to keep target stderr clean
+              log.debug("Class '{}' verification failed: {}", c.getName(), e.toString());
               sendCommand(
                   new MessageCommand(
                       "[BTRACE WARN] Class verification failed: "
@@ -544,13 +626,14 @@ boolean retransformLoaded() throws UnmodifiableClassException {
               try {
                 inst.retransformClasses(c);
               } catch (ClassFormatError | VerifyError e1) {
-                log.debug("Class '{}' verification failed", c.getName(), e);
+                // Avoid printing full stack traces in debug to keep target stderr clean
+                log.debug("Class '{}' verification failed: {}", c.getName(), e1.toString());
                 sendCommand(
                     new MessageCommand(
                         "[BTRACE WARN] Class verification failed: "
                             + c.getName()
                             + " ("
-                            + e.getMessage()
+                            + e1.getMessage()
                             + ")"));
               }
             }