Skip to content

Commit ce9547c

Browse files
committed
fix: recursive iteration on the local directory
1 parent 0f12c7f commit ce9547c

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

  • src/main/java/org/commoncrawl/whirlwind

src/main/java/org/commoncrawl/whirlwind/Duck.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -154,7 +154,7 @@ private static List<String> getLocalParquetFiles(Path indexPath) throws IOExcept
154154
System.exit(1);
155155
}
156156

157-
List<String> files = Files.list(indexPath).filter(p -> p.toString().endsWith(".parquet")).map(Path::toString)
157+
List<String> files = Files.walk(indexPath).map(Path::toString).filter(string -> string.endsWith(".parquet"))
158158
.collect(Collectors.toList());
159159

160160
if (files.isEmpty()) {

0 commit comments

Comments
 (0)