Skip to content

Commit fd67186

Browse files
authored
feat: Wire up IndexSource splits to Spark task sources (#27817)
Summary: `findTableScanNodes()` does not match `IndexSourceNode`, so index splits never reach Spark task assignments. `IndexLookupJoin` sees zero index splits and produces no matches -- inner joins return empty results, and left joins silently fill the right side with NULLs. Fix: extend the predicate to also match `IndexSourceNode`. ## Release Notes ``` == NO RELEASE NOTE == ```
1 parent 65547f4 commit fd67186

1 file changed

Lines changed: 2 additions & 1 deletion

File tree

presto-spark-base/src/main/java/com/facebook/presto/spark/planner/PrestoSparkRddFactory.java

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@
3737
import com.facebook.presto.spark.classloader_interface.SerializedTaskInfo;
3838
import com.facebook.presto.spi.PrestoException;
3939
import com.facebook.presto.spi.WarningCollector;
40+
import com.facebook.presto.spi.plan.IndexSourceNode;
4041
import com.facebook.presto.spi.plan.PartitioningHandle;
4142
import com.facebook.presto.spi.plan.PlanFragmentId;
4243
import com.facebook.presto.spi.plan.PlanNode;
@@ -369,7 +370,7 @@ private ListMultimap<Integer, SerializedPrestoSparkTaskSource> createTaskSources
369370
private static List<PrestoSparkSource> findTableScanNodes(PlanNode node)
370371
{
371372
return searchFrom(node)
372-
.where(TableScanNode.class::isInstance)
373+
.where(n -> n instanceof TableScanNode || n instanceof IndexSourceNode)
373374
.findAll().stream().map(t -> new PrestoSparkSource(t.getId(), t)).collect(Collectors.toList());
374375
}
375376

0 commit comments

Comments
 (0)