Commit 8a6555e
fix: Handle trailing tabs and quoted fields in Agilent Gen5 parser
Some Gen5 exports (e.g. Synergy H1) produce files with trailing tabs on
every line, leading tabs on data sections, and quoted fields containing
commas. This caused multiple parse failures: header DataFrame had extra
rows, section names were quoted, kinetic/results data had shifted columns,
and non-UTF-8 encoding wasn't detected.
Fixes:
- Add chardet encoding detection for non-UTF-8 files
- Strip trailing tabs from all lines at read time
- Strip quotes from procedure detail lines in DeviceControlData
- Extract section names from first non-empty tab field (handles leading tabs)
- Detect and strip extra leading tab from data sections
- Merge section data when a blank line separates header from content
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>1 parent 4307c74 commit 8a6555e
3 files changed
Lines changed: 47 additions & 9 deletions
File tree
- src/allotropy/parsers/agilent_gen5
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| 8 | + | |
8 | 9 | | |
9 | 10 | | |
10 | 11 | | |
| |||
35 | 36 | | |
36 | 37 | | |
37 | 38 | | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
38 | 45 | | |
39 | 46 | | |
40 | 47 | | |
| |||
Lines changed: 39 additions & 8 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
23 | | - | |
| 23 | + | |
24 | 24 | | |
25 | 25 | | |
26 | 26 | | |
| |||
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
34 | | - | |
| 34 | + | |
| 35 | + | |
35 | 36 | | |
36 | 37 | | |
37 | 38 | | |
| |||
75 | 76 | | |
76 | 77 | | |
77 | 78 | | |
78 | | - | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
79 | 87 | | |
80 | 88 | | |
81 | 89 | | |
| |||
87 | 95 | | |
88 | 96 | | |
89 | 97 | | |
90 | | - | |
| 98 | + | |
91 | 99 | | |
92 | 100 | | |
93 | 101 | | |
94 | 102 | | |
95 | 103 | | |
96 | | - | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
97 | 107 | | |
98 | 108 | | |
99 | 109 | | |
100 | 110 | | |
101 | | - | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
102 | 114 | | |
103 | 115 | | |
104 | | - | |
| 116 | + | |
105 | 117 | | |
106 | | - | |
| 118 | + | |
107 | 119 | | |
108 | 120 | | |
109 | 121 | | |
110 | 122 | | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
111 | 142 | | |
112 | 143 | | |
113 | 144 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
240 | 240 | | |
241 | 241 | | |
242 | 242 | | |
243 | | - | |
| 243 | + | |
244 | 244 | | |
245 | 245 | | |
246 | 246 | | |
| |||
0 commit comments