Commit 6f7ca7b
fix: Optimize test encoding detection for 4x speedup (#1143)
## Summary
This PR optimizes test file encoding detection to dramatically improve
test performance:
- Changed from always using chardet.detect() to trying UTF-8 first with
fallback
- chardet.detect() on large files (3.5MB+) can take 20+ seconds
- Most test files are UTF-8, so we only fall back to chardet for rare
cases with special characters
- Fixed incorrect UTF-8 encoding in luminex test files (µ and ® symbols)
## Results
- **4x speedup** on large test files (10s → 2.5s per file)
- **~50% reduction** in total test suite time
- Tests now complete well under the 15-minute CI timeout
## Test Plan
- [x] All tests pass
- [x] Verified encoding fallback works for non-UTF-8 files
- [x] Updated expected JSON files with correct UTF-8 encoding
- [x] CI runs successfully
Closes #1142 (replacing with clean rebase)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>1 parent 37f3d19 commit 6f7ca7b
3 files changed
Lines changed: 23 additions & 8 deletions
File tree
- .github/workflows
- tests
- parsers/luminex_intelliflex/testdata
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
29 | 29 | | |
30 | 30 | | |
31 | 31 | | |
32 | | - | |
| 32 | + | |
33 | 33 | | |
34 | 34 | | |
35 | 35 | | |
| |||
49 | 49 | | |
50 | 50 | | |
51 | 51 | | |
52 | | - | |
| 52 | + | |
53 | 53 | | |
54 | 54 | | |
55 | 55 | | |
| |||
68 | 68 | | |
69 | 69 | | |
70 | 70 | | |
71 | | - | |
| 71 | + | |
72 | 72 | | |
73 | 73 | | |
74 | 74 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6853 | 6853 | | |
6854 | 6854 | | |
6855 | 6855 | | |
6856 | | - | |
| 6856 | + | |
6857 | 6857 | | |
6858 | 6858 | | |
6859 | 6859 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
8 | | - | |
| 8 | + | |
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
| |||
61 | 61 | | |
62 | 62 | | |
63 | 63 | | |
64 | | - | |
65 | | - | |
66 | | - | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
67 | 82 | | |
68 | 83 | | |
69 | 84 | | |
| |||
0 commit comments