Skip to content

Improve path_rel performance#520

Open
orgadish wants to merge 6 commits into
r-lib:mainfrom
orgadish:og/519_path_rel_performance
Open

Improve path_rel performance#520
orgadish wants to merge 6 commits into
r-lib:mainfrom
orgadish:og/519_path_rel_performance

Conversation

@orgadish

@orgadish orgadish commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

#519

Improved path_rel performance by:

  1. Hoisting path_split(start) out of the loop entirely.
  2. Performing path_split(path) in a single vectorized call outside the R loop.
  3. Adding a new, internal path_common_parts that takes the components, rather than having to split again, and then passing the vectorized-split parts directly.
  4. The internal path_common_parts uses vectorized calls rather than R loops.
  5. Deduplicating path input to path_rel.

Performance Improvement:

  • ~10x improvement on 5k vector with no duplication (1.77 -> 200ms)
  • ~350x improvement on 5k vector with 50x duplication (2.09s -> 6ms)

Tests pass:

[ FAIL 0 | WARN 0 | SKIP 5 | PASS 280 ]

── Skipped tests (5) ────────────────────────────────────────────────────────────────────────────────────────────────────────
• On Windows (5): test-path.R:118:5, test-path.R:129:5, test-path.R:148:5, test-path.R:161:5, test-path.R:716:5

Benchmark

# Load dev path_rel and path_common
devtools::load_all("D:/Github/fs")
#> ℹ Loading fs

# Original path_rel and path_common
original_path_common <- function(path) {
  ...  # Copied from fs
}
original_path_rel <- function(path, start = ".") {
  ...  # Copied from fs, except using original_path_common()
}

all_3_letter_combos <- do.call(paste0, expand.grid(rep(list(LETTERS), 3)))

set.seed(3)
top_path <- "x/y/"

# Case 1: No duplication
paths <- paste0(top_path, sample(all_3_letter_combos, 5000), "/file.csv")
str(paths)
#>  chr [1:5000] "x/y/OHQ/file.csv" "x/y/JXJ/file.csv" "x/y/DDK/file.csv" ...

## path_rel()
microbenchmark::microbenchmark(
  new = path_rel(paths, start=top_path),
  original = original_path_rel(paths, start=top_path),
  times = 10L
)
#> Unit: milliseconds
#>      expr       min        lq      mean    median        uq      max neval cld
#>       new  181.1754  183.2254  256.2644  200.0344  344.2702  383.949    10  a 
#>  original 1723.4413 1757.3969 1920.8330 1769.0271 1786.6630 2693.200    10   b

## path_common()
microbenchmark::microbenchmark(
  new = path_common(paths),
  original = original_path_common(paths),
  times = 10L
)
#> Unit: milliseconds
#>      expr     min      lq     mean  median      uq     max neval cld
#>       new 71.0870 72.2124 79.92248 79.7240 85.2738 97.9682    10   a
#>  original 76.5306 77.0471 82.81926 83.1001 88.1458 89.9067    10   a


# Case 2: With duplication

# Create a list with significant duplication (50x duplication).
dup_paths <- sample(paths, length(paths) / 50) |> 
  rep(50) |> 
  sample()  # Reshuffle to ensure no sort-based benefits.
str(dup_paths)
#>  chr [1:5000] "x/y/EBC/file.csv" "x/y/ZUI/file.csv" "x/y/XSC/file.csv" ...

## path_rel()
microbenchmark::microbenchmark(
  new = path_rel(dup_paths, start=top_path),
  original = original_path_rel(dup_paths, start=top_path),
  deduped_og = deduped::deduped(original_path_rel)(dup_paths, start=top_path),
  deduped_new = deduped::deduped(path_rel)(dup_paths, start=top_path),
  times = 10L
)
#> Unit: milliseconds
#>         expr       min        lq       mean     median        uq       max neval cld
#>          new    4.9530    6.0521    6.29429    6.32245    6.3719    8.5303    10  a 
#>     original 1883.7256 1989.1657 2137.89882 2092.36985 2314.7633 2428.3711    10   b
#>   deduped_og   34.9242   39.0378   44.23877   40.64605   42.8788   68.3704    10  a 
#>  deduped_new    4.7585    5.6040   17.32869    5.80520    6.1606  119.5408    10  a 

## path_common()
microbenchmark::microbenchmark(
  new = path_common(dup_paths),
  original = original_path_common(dup_paths),
  times = 10L
)
#> Unit: milliseconds
#>      expr     min      lq     mean   median      uq     max neval cld
#>       new 72.1070 72.7383 76.19327 74.94015 79.1212 84.9619    10  a 
#>  original 73.5073 76.5788 87.99847 91.37255 97.9526 99.3352    10   b

Created on 2026-07-04 with reprex v2.1.1

orgadish added 6 commits July 4, 2026 22:20
1. Hoisting split of start out of loop.
2. Splitting all paths in a single vectorized call outside the loop.
3. Passing the split list into an internal version of path_common (`path_common_parts`) that takes the components, rather than having to split again.
4. The internal `path_common_parts` uses vectorized calls rather than R loops.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant