[SPARK-57695][CORE] Make TestUtils.recursiveList null-safe by reusing Utils.recursiveList

[LuciferYang]

What changes were proposed in this pull request?

TestUtils.recursiveList duplicated the recursive directory walk already provided by Utils.recursiveList (i.e. SparkFileUtils.recursiveList). This PR removes the duplicated copy and delegates to Utils.recursiveList:

// before
def recursiveList(f: File): Array[File] = {
  require(f.isDirectory)
  val current = f.listFiles
  current ++ current.filter(_.isDirectory).flatMap(recursiveList)
}

// after
def recursiveList(f: File): Array[File] = Utils.recursiveList(f)

Why are the changes needed?

The duplicated implementation carried the same two issues that were just fixed in SparkFileUtils.recursiveList under SPARK-57530:

1. It called File.listFiles without a null check, so an IO error (or the directory being removed mid-walk) would throw an NPE.
2. The current ++ ... .flatMap(...) form had no linear-time guarantee.

By delegating to Utils.recursiveList, TestUtils.recursiveList automatically picks up the null-safety (a directory that cannot be listed is skipped with a warning instead of throwing) and the O(n) traversal from SPARK-57530, and the duplicated logic is removed.

This is a follow-up to SPARK-57530, which is already merged to master.

Does this PR introduce any user-facing change?

No. TestUtils is a test-only private[spark] helper; this is an internal refactor with no behavior change for any successful directory walk.

How was this patch tested?

Existing tests that use TestUtils.recursiveList continue to exercise it. The behavioral contract for a readable directory tree is unchanged (same set of files returned); the only difference is that an unreadable directory is now skipped with a warning rather than throwing, matching Utils.recursiveList.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code

[MaxGekk]

0 blocking, 0 non-blocking, 0 nits.
Clean dedup — delegating TestUtils.recursiveList to the SPARK-57530-fixed Utils.recursiveList is behavior-preserving. LGTM.

Verification

• Identical result set + precondition: both return every entry (files and subdirectories) at every level, once; require(f.isDirectory) is preserved by the delegate, so the throw on a non-directory argument is unchanged.
• The only delta — DFS → BFS order — is unobservable for every current caller: the order-sensitive ones filter to a single element before .head (JsonSuite via coalesce(1); PartitionedWriteSuite asserts files.length == 1 first; FileMetadataStructSuite reads the single .parquet); CSVSuite/TextSuite mirror JsonSuite; the rest use .count / .filter / set ops.
• Null-safety (unreadable dir → warn-and-skip instead of NPE, from SPARK-57530) is the intended improvement and isn't reached by the readable temp-dir test callers.
• The delegate has dedicated coverage in SparkFileUtilsSuite; no new test needed for a pure delegation.

[MaxGekk]

The failed CI is not related to the PR, I believe.

+1, LGTM. Merging to master/4.x.
Thank you, @LuciferYang.