feat(appsec): extend RASP callsite coverage to File-argument constructors of FileOutputStream and FileInputStream (#11113)

* feat(rasp): instrument FileOutputStream/FileInputStream(File) constructors

Add RASP callsite coverage for File-argument constructors that were
previously not instrumented:

- FileOutputStream(File) and FileOutputStream(File, boolean): call
  FileIORaspHelper.INSTANCE.beforeFileWritten(file.getPath())
- FileInputStream(File): call
  FileIORaspHelper.INSTANCE.beforeFileLoaded(file.getPath())

No IAST changes — the File-based constructors delegate path resolution
to the JVM, so IAST taint tracking via the String constructor already
covers those code paths at a higher level.

Tests added following the existing RASP test pattern.

* feat(appsec): extend RASP file I/O coverage to FileReader, FileWriter, RandomAccessFile, Files.* and FileChannel

Extends RASP callsite instrumentation (APPSEC-61874) beyond FileInputStream/FileOutputStream
to all remaining Java file I/O APIs that were not covered. No IAST changes.

New callsites:
- FileReaderCallSite: FileReader(String/File) + Java 11+ Charset variants → beforeFileLoaded
- FileWriterCallSite: FileWriter(String/File/boolean) + Java 11+ Charset variants → beforeFileWritten
- RandomAccessFileCallSite: RandomAccessFile(String/File, mode) → beforeFileLoaded for "r",
  both beforeFileLoaded + beforeFileWritten for "rw"/"rws"/"rwd"
- FilesCallSite: all Files.* read and write methods (newOutputStream, copy(IS,Path),
  write, writeString, newBufferedWriter, move, newInputStream, readAllBytes, readAllLines,
  readString, newBufferedReader, lines)
- FileChannelCallSite: FileChannel.open(Path, ...) → fires both read and write callbacks

Extended callsites:
- PathCallSite: add resolve(Path) and resolveSibling(Path) → beforeFileLoaded
- PathsCallSite: add Path.of(String[], URI) (Java 11+) → beforeFileLoaded

FileIORaspHelper: add beforeRandomAccessFileOpened(path, mode) helper

Relates to #11084 and #11113

* Apply spotless formatting

* Add Java 11 test suite for RASP coverage of Java 11 file I/O APIs

Adds a java11Test source set that compiles with --release 11 and runs
only on JDK 11+. Tests cover the Java 11-only overloads that were
instrumented but previously untestable from Java 8 sources:

- FileReader(String/File, Charset) constructors
- FileWriter(String/File, Charset[, boolean]) constructors
- Files.writeString(Path, CharSequence, [Charset,] OpenOption...)
- Files.readString(Path[, Charset])
- Path.of(String, String[]) and Path.of(URI) static methods

Build configuration uses ext.java11TestMinJavaVersionForTests so the
testJvmConstraints plugin skips the suite on pre-11 JVMs.

* Remove unused StandardCopyOption import from FilesCallSiteTest

* fix(appsec): gate FileChannel write callback on write-capable open options

FileChannel.open() with READ-only options was incorrectly triggering the
fileWritten callback, which could cause false positives in the zipslip
rule (dog-920-110) when a read-only channel open with a traversal path
coincided with a multipart zip upload in the same request.

Split beforeOpen into two overload-specific methods so the OpenOption
arguments can be inspected at the call site, mirroring the existing
pattern in beforeRandomAccessFileOpened.

Also fix a latent bug in AdviceGeneratorImpl: .sorted() without a
comparator on ArgumentSpecification (which does not implement Comparable)
would ClassCastException when an advice method captures a strict subset
of a target method's arguments. Fixed with Comparator.comparingInt.

* fix(appsec): capture all FileChannel.open args to avoid CSI generator partial-arg path

beforeOpenSet previously captured only 2 of 3 arguments, triggering the
partial-argument code path in AdviceGeneratorImpl which calls Stream.sorted()
without a comparator on ArgumentSpecification (not Comparable). Adding the
unused FileAttribute[] third parameter makes the capture complete and
sequential, so isPositionalArguments() returns false and the sorted() path
is never entered.

Author: jandro996

dd-java-agent/instrumentation/java/java-io-1.8/build.gradle

@@ -8,8 +8,20 @@ apply from: "$rootDir/gradle/java.gradle"
 apply plugin: 'dd-trace-java.call-site-instrumentation'
 
 addTestSuiteForDir('latestDepTest', 'test')
+addTestSuiteForDir('java11Test', 'java11Test')
+
+// java11Test only runs on JDK 11+; the testJvmConstraints plugin reads this property by convention
+ext.java11TestMinJavaVersionForTests = JavaVersion.VERSION_11
+
+tasks.named("compileJava11TestJava", JavaCompile) {
+  configureCompiler(it, 11)
+}
+tasks.named("compileJava11TestGroovy", GroovyCompile) {
+  configureCompiler(it, 11)
+}
 
 dependencies {
   testRuntimeOnly project(':dd-java-agent:instrumentation:datadog:asm:iast-instrumenter')
   testImplementation group: 'org.apache.tomcat', name: 'tomcat-catalina', version: '9.0.56'
+  java11TestImplementation sourceSets.test.output
 }

dd-java-agent/instrumentation/java/java-io-1.8/src/java11Test/groovy/datadog/trace/instrumentation/java/io/FileReaderCharsetCallSiteTest.groovy

@@ -0,0 +1,35 @@
+package datadog.trace.instrumentation.java.io
+
+import datadog.trace.instrumentation.java.lang.FileIORaspHelper
+import foo.bar.TestFileReaderCharsetSuite
+
+import java.nio.charset.Charset
+
+class FileReaderCharsetCallSiteTest extends BaseIoRaspCallSiteTest {
+
+  void 'test RASP new FileReader with String path and Charset'() {
+    setup:
+    final helper = Mock(FileIORaspHelper)
+    FileIORaspHelper.INSTANCE = helper
+    final file = newFile('test_rasp_fr_str_cs.txt')
+
+    when:
+    TestFileReaderCharsetSuite.newFileReader(file.path, Charset.defaultCharset()).close()
+
+    then:
+    1 * helper.beforeFileLoaded(file.path)
+  }
+
+  void 'test RASP new FileReader with File and Charset'() {
+    setup:
+    final helper = Mock(FileIORaspHelper)
+    FileIORaspHelper.INSTANCE = helper
+    final file = newFile('test_rasp_fr_file_cs.txt')
+
+    when:
+    TestFileReaderCharsetSuite.newFileReader(file, Charset.defaultCharset()).close()
+
+    then:
+    1 * helper.beforeFileLoaded(file.path)
+  }
+}

dd-java-agent/instrumentation/java/java-io-1.8/src/java11Test/groovy/datadog/trace/instrumentation/java/io/FileWriterCharsetCallSiteTest.groovy

@@ -0,0 +1,61 @@
+package datadog.trace.instrumentation.java.io
+
+import datadog.trace.instrumentation.java.lang.FileIORaspHelper
+import foo.bar.TestFileWriterCharsetSuite
+
+import java.nio.charset.Charset
+
+class FileWriterCharsetCallSiteTest extends BaseIoRaspCallSiteTest {
+
+  void 'test RASP new FileWriter with String path and Charset'() {
+    setup:
+    final helper = Mock(FileIORaspHelper)
+    FileIORaspHelper.INSTANCE = helper
+    final file = newFile('test_rasp_fw_str_cs.txt')
+
+    when:
+    TestFileWriterCharsetSuite.newFileWriter(file.path, Charset.defaultCharset()).close()
+
+    then:
+    1 * helper.beforeFileWritten(file.path)
+  }
+
+  void 'test RASP new FileWriter with String path, Charset, and append flag'() {
+    setup:
+    final helper = Mock(FileIORaspHelper)
+    FileIORaspHelper.INSTANCE = helper
+    final file = newFile('test_rasp_fw_str_cs_append.txt')
+
+    when:
+    TestFileWriterCharsetSuite.newFileWriter(file.path, Charset.defaultCharset(), false).close()
+
+    then:
+    1 * helper.beforeFileWritten(file.path)
+  }
+
+  void 'test RASP new FileWriter with File and Charset'() {
+    setup:
+    final helper = Mock(FileIORaspHelper)
+    FileIORaspHelper.INSTANCE = helper
+    final file = newFile('test_rasp_fw_file_cs.txt')
+
+    when:
+    TestFileWriterCharsetSuite.newFileWriter(file, Charset.defaultCharset()).close()
+
+    then:
+    1 * helper.beforeFileWritten(file.path)
+  }
+
+  void 'test RASP new FileWriter with File, Charset, and append flag'() {
+    setup:
+    final helper = Mock(FileIORaspHelper)
+    FileIORaspHelper.INSTANCE = helper
+    final file = newFile('test_rasp_fw_file_cs_append.txt')
+
+    when:
+    TestFileWriterCharsetSuite.newFileWriter(file, Charset.defaultCharset(), false).close()
+
+    then:
+    1 * helper.beforeFileWritten(file.path)
+  }
+}

dd-java-agent/instrumentation/java/java-io-1.8/src/java11Test/groovy/datadog/trace/instrumentation/java/io/FilesJava11CallSiteTest.groovy

@@ -0,0 +1,61 @@
+package datadog.trace.instrumentation.java.io
+
+import datadog.trace.instrumentation.java.lang.FileIORaspHelper
+import foo.bar.TestFilesJava11Suite
+
+import java.nio.charset.StandardCharsets
+
+class FilesJava11CallSiteTest extends BaseIoRaspCallSiteTest {
+
+  void 'test RASP Files.writeString without charset'() {
+    setup:
+    final helper = Mock(FileIORaspHelper)
+    FileIORaspHelper.INSTANCE = helper
+    final path = temporaryFolder.resolve('test_rasp_writestring.txt')
+
+    when:
+    TestFilesJava11Suite.writeString(path, 'hello')
+
+    then:
+    1 * helper.beforeFileWritten(path.toString())
+  }
+
+  void 'test RASP Files.writeString with charset'() {
+    setup:
+    final helper = Mock(FileIORaspHelper)
+    FileIORaspHelper.INSTANCE = helper
+    final path = temporaryFolder.resolve('test_rasp_writestring_cs.txt')
+
+    when:
+    TestFilesJava11Suite.writeString(path, 'hello', StandardCharsets.UTF_8)
+
+    then:
+    1 * helper.beforeFileWritten(path.toString())
+  }
+
+  void 'test RASP Files.readString without charset'() {
+    setup:
+    final helper = Mock(FileIORaspHelper)
+    FileIORaspHelper.INSTANCE = helper
+    final path = newFile('test_rasp_readstring.txt').toPath()
+
+    when:
+    TestFilesJava11Suite.readString(path)
+
+    then:
+    1 * helper.beforeFileLoaded(path.toString())
+  }
+
+  void 'test RASP Files.readString with charset'() {
+    setup:
+    final helper = Mock(FileIORaspHelper)
+    FileIORaspHelper.INSTANCE = helper
+    final path = newFile('test_rasp_readstring_cs.txt').toPath()
+
+    when:
+    TestFilesJava11Suite.readString(path, StandardCharsets.UTF_8)
+
+    then:
+    1 * helper.beforeFileLoaded(path.toString())
+  }
+}

dd-java-agent/instrumentation/java/java-io-1.8/src/java11Test/groovy/datadog/trace/instrumentation/java/io/PathOfCallSiteTest.groovy

@@ -0,0 +1,37 @@
+package datadog.trace.instrumentation.java.io
+
+import datadog.trace.instrumentation.java.lang.FileIORaspHelper
+import foo.bar.TestPathOfSuite
+
+class PathOfCallSiteTest extends BaseIoRaspCallSiteTest {
+
+  void 'test RASP Path.of from strings'(final String first, final String... more) {
+    setup:
+    final helper = Mock(FileIORaspHelper)
+    FileIORaspHelper.INSTANCE = helper
+
+    when:
+    TestPathOfSuite.of(first, more)
+
+    then:
+    1 * helper.beforeFileLoaded(first, more)
+
+    where:
+    first      | more
+    'test.txt' | [] as String[]
+    '/tmp'     | ['log', 'test.txt'] as String[]
+  }
+
+  void 'test RASP Path.of from URI'() {
+    setup:
+    final helper = Mock(FileIORaspHelper)
+    FileIORaspHelper.INSTANCE = helper
+    final uri = new URI('file:/test.txt')
+
+    when:
+    TestPathOfSuite.of(uri)
+
+    then:
+    1 * helper.beforeFileLoaded(uri)
+  }
+}

dd-java-agent/instrumentation/java/java-io-1.8/src/java11Test/java/foo/bar/TestFileReaderCharsetSuite.java

@@ -0,0 +1,19 @@
+package foo.bar;
+
+import java.io.File;
+import java.io.FileReader;
+import java.io.IOException;
+import java.nio.charset.Charset;
+
+public class TestFileReaderCharsetSuite {
+
+  public static FileReader newFileReader(final String path, final Charset charset)
+      throws IOException {
+    return new FileReader(path, charset);
+  }
+
+  public static FileReader newFileReader(final File file, final Charset charset)
+      throws IOException {
+    return new FileReader(file, charset);
+  }
+}

dd-java-agent/instrumentation/java/java-io-1.8/src/java11Test/java/foo/bar/TestFileWriterCharsetSuite.java

@@ -0,0 +1,29 @@
+package foo.bar;
+
+import java.io.File;
+import java.io.FileWriter;
+import java.io.IOException;
+import java.nio.charset.Charset;
+
+public class TestFileWriterCharsetSuite {
+
+  public static FileWriter newFileWriter(final String path, final Charset charset)
+      throws IOException {
+    return new FileWriter(path, charset);
+  }
+
+  public static FileWriter newFileWriter(
+      final String path, final Charset charset, final boolean append) throws IOException {
+    return new FileWriter(path, charset, append);
+  }
+
+  public static FileWriter newFileWriter(final File file, final Charset charset)
+      throws IOException {
+    return new FileWriter(file, charset);
+  }
+
+  public static FileWriter newFileWriter(
+      final File file, final Charset charset, final boolean append) throws IOException {
+    return new FileWriter(file, charset, append);
+  }
+}

dd-java-agent/instrumentation/java/java-io-1.8/src/java11Test/java/foo/bar/TestFilesJava11Suite.java

@@ -0,0 +1,32 @@
+package foo.bar;
+
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.file.Files;
+import java.nio.file.OpenOption;
+import java.nio.file.Path;
+
+public class TestFilesJava11Suite {
+
+  public static Path writeString(
+      final Path path, final CharSequence content, final OpenOption... options) throws IOException {
+    return Files.writeString(path, content, options);
+  }
+
+  public static Path writeString(
+      final Path path,
+      final CharSequence content,
+      final Charset charset,
+      final OpenOption... options)
+      throws IOException {
+    return Files.writeString(path, content, charset, options);
+  }
+
+  public static String readString(final Path path) throws IOException {
+    return Files.readString(path);
+  }
+
+  public static String readString(final Path path, final Charset charset) throws IOException {
+    return Files.readString(path, charset);
+  }
+}

dd-java-agent/instrumentation/java/java-io-1.8/src/java11Test/java/foo/bar/TestPathOfSuite.java

@@ -0,0 +1,15 @@
+package foo.bar;
+
+import java.net.URI;
+import java.nio.file.Path;
+
+public class TestPathOfSuite {
+
+  public static Path of(final String first, final String... more) {
+    return Path.of(first, more);
+  }
+
+  public static Path of(final URI uri) {
+    return Path.of(uri);
+  }
+}

dd-java-agent/instrumentation/java/java-io-1.8/src/main/java/datadog/trace/instrumentation/java/lang/FileChannelCallSite.java

@@ -0,0 +1,77 @@
+package datadog.trace.instrumentation.java.lang;
+
+import datadog.trace.agent.tooling.csi.CallSite;
+import datadog.trace.api.appsec.RaspCallSites;
+import java.nio.file.OpenOption;
+import java.nio.file.Path;
+import java.nio.file.StandardOpenOption;
+import java.nio.file.attribute.FileAttribute;
+import java.util.Set;
+import javax.annotation.Nullable;
+
+@CallSite(
+    spi = {RaspCallSites.class},
+    helpers = FileIORaspHelper.class)
+public class FileChannelCallSite {
+
+  @CallSite.Before(
+      "java.nio.channels.FileChannel java.nio.channels.FileChannel.open(java.nio.file.Path, java.nio.file.OpenOption[])")
+  public static void beforeOpenArray(
+      @CallSite.Argument(0) @Nullable final Path path,
+      @CallSite.Argument(1) @Nullable final OpenOption[] options) {
+    if (path != null) {
+      String pathStr = path.toString();
+      FileIORaspHelper.INSTANCE.beforeFileLoaded(pathStr);
+      if (hasWriteOption(options)) {
+        FileIORaspHelper.INSTANCE.beforeFileWritten(pathStr);
+      }
+    }
+  }
+
+  @CallSite.Before(
+      "java.nio.channels.FileChannel java.nio.channels.FileChannel.open(java.nio.file.Path, java.util.Set, java.nio.file.attribute.FileAttribute[])")
+  public static void beforeOpenSet(
+      @CallSite.Argument(0) @Nullable final Path path,
+      @CallSite.Argument(1) @Nullable final Set<? extends OpenOption> options,
+      @CallSite.Argument(2) @Nullable final FileAttribute<?>[] attrs) {
+    if (path != null) {
+      String pathStr = path.toString();
+      FileIORaspHelper.INSTANCE.beforeFileLoaded(pathStr);
+      if (hasWriteOption(options)) {
+        FileIORaspHelper.INSTANCE.beforeFileWritten(pathStr);
+      }
+    }
+  }
+
+  private static boolean hasWriteOption(@Nullable final OpenOption[] options) {
+    if (options == null) {
+      return false;
+    }
+    for (OpenOption opt : options) {
+      if (isWriteOption(opt)) {
+        return true;
+      }
+    }
+    return false;
+  }
+
+  private static boolean hasWriteOption(@Nullable final Set<? extends OpenOption> options) {
+    if (options == null) {
+      return false;
+    }
+    for (OpenOption opt : options) {
+      if (isWriteOption(opt)) {
+        return true;
+      }
+    }
+    return false;
+  }
+
+  private static boolean isWriteOption(final OpenOption opt) {
+    return opt == StandardOpenOption.WRITE
+        || opt == StandardOpenOption.APPEND
+        || opt == StandardOpenOption.CREATE
+        || opt == StandardOpenOption.CREATE_NEW
+        || opt == StandardOpenOption.TRUNCATE_EXISTING;
+  }
+}