Rewrite matchFiles to not use regexp2, remove other regexp uses

[jakebailey]

This is a total redo of #1483.

After this PR, our use of regexp/regexp2 is limited to:

• Tests
• UnquoteString (which is documented as being implemented incorrectly)
• semver parsing
• Some complicated code in modulespecifiers to handle auto-import regexes (probably not removable).

I have also tried to make a concurrent version like what tsgolint patches in, but it's a net negative in the new code. Hopefully this means it's not needed? (EDIT: they confirmed so.)

goos: linux
goarch: amd64
pkg: github.com/microsoft/typescript-go/internal/vfs/vfsmatch
cpu: Intel(R) Core(TM) i9-10900K CPU @ 3.70GHz
                                    │    old.txt    │               new.txt               │
                                    │    sec/op     │   sec/op     vs base                │
ReadDirectory/LiteralIncludes          17.521µ ± 4%   3.551µ ± 2%  -79.74% (p=0.000 n=10)
ReadDirectory/WildcardIncludes         37.124µ ± 1%   6.970µ ± 2%  -81.23% (p=0.000 n=10)
ReadDirectory/RecursiveWildcard        31.154µ ± 3%   4.324µ ± 1%  -86.12% (p=0.000 n=10)
ReadDirectory/RecursiveWithExcludes    56.050µ ± 2%   6.771µ ± 4%  -87.92% (p=0.000 n=10)
ReadDirectory/ComplexPattern            67.84µ ± 1%   17.11µ ± 4%  -74.79% (p=0.000 n=10)
ReadDirectory/DottedFolders            32.260µ ± 2%   6.249µ ± 1%  -80.63% (p=0.000 n=10)
ReadDirectory/CommonPackageFolders     15.027µ ± 1%   2.614µ ± 2%  -82.61% (p=0.000 n=10)
ReadDirectory/NoIncludes                5.631µ ± 2%   2.446µ ± 1%  -56.56% (p=0.000 n=10)
ReadDirectory/MultipleRecursive        31.315µ ± 2%   4.938µ ± 1%  -84.23% (p=0.000 n=10)
ReadDirectory/LargeFileSystem         1438.08µ ± 1%   68.38µ ± 1%  -95.24% (p=0.000 n=10)
ReadDirectory/LargeAllFiles           1892.50µ ± 2%   93.90µ ± 1%  -95.04% (p=0.000 n=10)
geomean                                 56.28µ        8.451µ       -84.98%

                                    │    old.txt    │               new.txt                │
                                    │     B/op      │     B/op      vs base                │
ReadDirectory/LiteralIncludes          4.555Ki ± 0%   2.133Ki ± 0%  -53.17% (p=0.000 n=10)
ReadDirectory/WildcardIncludes         8.368Ki ± 0%   3.914Ki ± 0%  -53.23% (p=0.000 n=10)
ReadDirectory/RecursiveWildcard        4.469Ki ± 0%   1.531Ki ± 0%  -65.73% (p=0.000 n=10)
ReadDirectory/RecursiveWithExcludes    7.024Ki ± 0%   2.953Ki ± 0%  -57.96% (p=0.000 n=10)
ReadDirectory/ComplexPattern          17.627Ki ± 0%   9.758Ki ± 0%  -44.64% (p=0.000 n=10)
ReadDirectory/DottedFolders            6.414Ki ± 0%   2.531Ki ± 0%  -60.54% (p=0.000 n=10)
ReadDirectory/CommonPackageFolders     3.508Ki ± 0%   1.391Ki ± 0%  -60.36% (p=0.000 n=10)
ReadDirectory/NoIncludes               1.758Ki ± 0%   1.211Ki ± 0%  -31.11% (p=0.000 n=10)
ReadDirectory/MultipleRecursive        6.250Ki ± 0%   2.078Ki ± 0%  -66.75% (p=0.000 n=10)
ReadDirectory/LargeFileSystem          72.22Ki ± 0%   12.06Ki ± 0%  -83.30% (p=0.000 n=10)
ReadDirectory/LargeAllFiles           105.44Ki ± 0%   21.21Ki ± 0%  -79.89% (p=0.000 n=10)
geomean                                9.194Ki        3.454Ki       -62.44%

                                    │   old.txt   │              new.txt               │
                                    │  allocs/op  │ allocs/op   vs base                │
ReadDirectory/LiteralIncludes         154.00 ± 0%   38.00 ± 0%  -75.32% (p=0.000 n=10)
ReadDirectory/WildcardIncludes        253.00 ± 0%   82.00 ± 0%  -67.59% (p=0.000 n=10)
ReadDirectory/RecursiveWildcard       142.00 ± 0%   36.00 ± 0%  -74.65% (p=0.000 n=10)
ReadDirectory/RecursiveWithExcludes   185.00 ± 0%   55.00 ± 0%  -70.27% (p=0.000 n=10)
ReadDirectory/ComplexPattern           567.0 ± 0%   154.0 ± 0%  -72.84% (p=0.000 n=10)
ReadDirectory/DottedFolders           140.00 ± 0%   51.00 ± 0%  -63.57% (p=0.000 n=10)
ReadDirectory/CommonPackageFolders     98.00 ± 0%   27.00 ± 0%  -72.45% (p=0.000 n=10)
ReadDirectory/NoIncludes               62.00 ± 0%   36.00 ± 0%  -41.94% (p=0.000 n=10)
ReadDirectory/MultipleRecursive       151.00 ± 0%   40.00 ± 0%  -73.51% (p=0.000 n=10)
ReadDirectory/LargeFileSystem         1024.0 ± 0%   191.0 ± 0%  -81.35% (p=0.000 n=10)
ReadDirectory/LargeAllFiles           1254.0 ± 0%   334.0 ± 0%  -73.37% (p=0.000 n=10)
geomean                                230.1        66.90       -70.92%

[Copilot]

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated 4 comments.

[jakebailey]

I think that this is ready. As an exercise, I had copilot turn the code into a spec, deleted the code, then had a new session implement it from scratch. It came up with something different, which still passed the tests.

There's some differences between the old regex based one, but I think none that matter in practice. I'm fully prepared to delete the old implementation from this PR entirely, as I think it is net better, both in performance and in intentional design.

[camc314]

This is awesome!