Commit 8e25d7d
committed
feat: replace chunked Rounds 2-4 with isolated Rayon pool partitions
Instead of processing tables in chunks on the global Rayon pool
(where cross-table work-stealing causes contention), partition
tables into S groups using greedy LPT scheduling and give each
group its own isolated Rayon ThreadPool with cores/S threads.
Key changes:
- shard_parallelism(): configurable S (default cores/8, env SHARD_PARALLELISM)
- partition_tables_by_cost(): greedy LPT by rows * columns
- std::thread::scope + pool.install() for per-partition isolation
- SendPtr for disjoint transcript access across partitions
- Consumes cached LDEs via build_round1 (no recomputation)1 parent 3f45c4f commit 8e25d7d
1 file changed
Lines changed: 17 additions & 19 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1905 | 1905 | | |
1906 | 1906 | | |
1907 | 1907 | | |
1908 | | - | |
| 1908 | + | |
| 1909 | + | |
| 1910 | + | |
| 1911 | + | |
| 1912 | + | |
| 1913 | + | |
1909 | 1914 | | |
1910 | 1915 | | |
1911 | 1916 | | |
| |||
1926 | 1931 | | |
1927 | 1932 | | |
1928 | 1933 | | |
1929 | | - | |
1930 | | - | |
| 1934 | + | |
1931 | 1935 | | |
1932 | 1936 | | |
1933 | 1937 | | |
| |||
1945 | 1949 | | |
1946 | 1950 | | |
1947 | 1951 | | |
1948 | | - | |
1949 | | - | |
| 1952 | + | |
1950 | 1953 | | |
1951 | | - | |
1952 | | - | |
1953 | | - | |
| 1954 | + | |
| 1955 | + | |
1954 | 1956 | | |
1955 | 1957 | | |
1956 | 1958 | | |
| |||
1963 | 1965 | | |
1964 | 1966 | | |
1965 | 1967 | | |
1966 | | - | |
1967 | | - | |
| 1968 | + | |
| 1969 | + | |
| 1970 | + | |
1968 | 1971 | | |
1969 | 1972 | | |
1970 | 1973 | | |
| |||
2000 | 2003 | | |
2001 | 2004 | | |
2002 | 2005 | | |
2003 | | - | |
2004 | | - | |
2005 | | - | |
2006 | | - | |
| 2006 | + | |
2007 | 2007 | | |
2008 | 2008 | | |
2009 | 2009 | | |
| |||
2042 | 2042 | | |
2043 | 2043 | | |
2044 | 2044 | | |
2045 | | - | |
2046 | | - | |
| 2045 | + | |
2047 | 2046 | | |
2048 | 2047 | | |
2049 | 2048 | | |
| |||
2064 | 2063 | | |
2065 | 2064 | | |
2066 | 2065 | | |
2067 | | - | |
2068 | | - | |
2069 | | - | |
| 2066 | + | |
| 2067 | + | |
2070 | 2068 | | |
2071 | 2069 | | |
2072 | 2070 | | |
| |||
0 commit comments