Skip to content

Commit fe88e2b

Browse files
feat: Add vendor auto-discovery from file contents (#1195)
## Summary - Add `vendor_from_file()` and `vendor_from_io()` public API functions that auto-detect the correct `Vendor` for a file based on its contents - Make `vendor_type` optional (`None` default) in `allotrope_from_file`, `allotrope_from_io`, `allotrope_model_from_file`, `allotrope_model_from_io` — when `None`, auto-discovers the vendor - Add `AllotropeVendorNotFoundError` exception raised when no parser matches - Implement `sniff()` classmethods on all 53 parsers with lightweight file-content checks (sheet names, header patterns, XML root tags, etc.) - Add `discover_vendor_test.py` with 309 parametrized tests verifying discovery against all existing test data files - Add vendor auto-discovery guidance to CLAUDE.md for new parser development ## Design `discover_vendor()` in `parser_factory.py` runs a multi-pass detection strategy: 1. **Filter by extension** — only parsers supporting the file's extension are candidates 2. **Sniff pass** — call `sniff()` on each candidate, collect all matches 3. **Single match** — return immediately 4. **Multiple matches** — try `create_data()` on each sniff match to disambiguate (e.g., Gen5 vs Gen5Image both sniff True for `.txt`, but only one parses successfully) 5. **No sniff matches** — try `create_data()` on all candidates as brute-force fallback 6. **Sniff matched but parse failed** — trust the sniff result (parse failure may be unrelated to discovery, e.g., encoding issues) Each parser's `sniff()` is a lightweight classmethod: unique extensions return `True`, shared extensions check distinctive content (header columns, sheet names, XML root tags, etc.). UTF-16 LE/BE encoded files are handled with BOM detection. **Indistinguishable parser pairs** (same file format, either parser works): Gen5/Gen5Image, xPONENT/IntelliFlex, VisionLite/GenesysOnBoard, EchoCherryPick/EchoPlateReformat, QuantStudio/QuantStudioD&A, MSDWorkbench/MethodicalMind. ## Test plan - [x] All 1119 existing parser tests pass - [x] 309 new discovery tests pass — one per test data file, verifying correct vendor detection - [x] Lint + mypy clean 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 8009b8b commit fe88e2b

59 files changed

Lines changed: 1293 additions & 15 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

CLAUDE.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,25 @@ Generated models use these naming patterns:
6969

7070
Quantity value types use abbreviated unit names: `TQuantityValueDegC`, `TQuantityValueMAU`, `TQuantityValueMmHg`, `TQuantityValueMicroLPermin`, `TQuantityValueM`, `TQuantityValueNM`, `TQuantityValueM1s1`, `TQuantityValueS1`, `TQuantityValueRU`, `TQuantityValueS`.
7171

72+
## Vendor Auto-Discovery
73+
74+
Each parser has a `sniff(cls, named_file_contents) -> bool` classmethod used by `discover_vendor()` to auto-detect the correct parser for a file. The discovery logic in `parser_factory.py`:
75+
76+
1. Filters candidates by file extension
77+
2. Calls `sniff()` on each — collects all matches
78+
3. Single match: returns immediately
79+
4. Multiple matches: tries `create_data()` on each to disambiguate
80+
5. No sniff matches: tries `create_data()` on all candidates as fallback
81+
6. If sniff matched but parse failed: trusts the sniff result
82+
83+
**When adding a new parser:**
84+
- Implement `sniff()` — check file headers, sheet names, XML root tags, etc. Keep it lightweight.
85+
- If the file extension is unique to this parser (e.g. `.blr`, `.rslt`), `return True` is fine.
86+
- For shared extensions (`.csv`, `.txt`, `.xlsx`), check distinctive content patterns.
87+
- **Check sniff logic of related parsers** that share the same extension — your new sniff must not false-positive on their test files, and theirs must not match yours.
88+
- Run `hatch run test_all.py3.10:pytest tests/discover_vendor_test.py -x -q` to verify.
89+
- Handle encoding: many files use UTF-16 LE (BOM `\xff\xfe`). Decode appropriately before text checks.
90+
7291
## Code Rules
7392

7493
- **No runtime imports.** All imports must be at module level. Never put `import` or `from ... import` inside a function body to work around circular dependencies — fix the dependency instead.

src/allotropy/exceptions.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,10 @@ class AllotropeConversionError(AllotropyError):
3333
...
3434

3535

36+
class AllotropeVendorNotFoundError(AllotropyError):
37+
...
38+
39+
3640
def list_values(values: Collection[Any] | type[Enum]) -> list[str]:
3741
return sorted([str(v.value if isinstance(v, Enum) else v) for v in values])
3842

src/allotropy/parser_factory.py

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,8 @@
55
from typing import Any
66

77
from allotropy.allotrope.path_util import ROOT_DIR
8+
from allotropy.exceptions import AllotropeVendorNotFoundError
9+
from allotropy.named_file_contents import NamedFileContents
810
from allotropy.parsers.agilent_gen5.agilent_gen5_parser import AgilentGen5Parser
911
from allotropy.parsers.agilent_gen5_image.agilent_gen5_image_parser import (
1012
AgilentGen5ImageParser,
@@ -296,6 +298,40 @@ def get_parser(
296298
}
297299

298300

301+
def discover_vendor(named_file_contents: NamedFileContents) -> Vendor:
302+
extension = named_file_contents.extension
303+
candidates = [v for v in Vendor if extension in v.supported_extensions]
304+
# Pass 1: collect all sniff matches.
305+
sniffed: list[Vendor] = []
306+
for vendor in candidates:
307+
parser_cls = _VENDOR_TO_PARSER[vendor]
308+
named_file_contents.contents.seek(0)
309+
try:
310+
if parser_cls.sniff(named_file_contents):
311+
sniffed.append(vendor)
312+
except Exception: # noqa: S112
313+
continue
314+
# Exactly one sniff match — return without try-parse.
315+
if len(sniffed) == 1:
316+
return sniffed[0]
317+
# Pass 2: multiple sniff matches — try parsing to disambiguate.
318+
# No sniff matches — try parsing all candidates as fallback.
319+
try_parse = sniffed or candidates
320+
for vendor in try_parse:
321+
named_file_contents.contents.seek(0)
322+
try:
323+
vendor.get_parser().create_data(named_file_contents)
324+
return vendor
325+
except Exception: # noqa: S112
326+
continue
327+
# If sniff matched but try-parse failed (e.g. encoding issues unrelated to
328+
# discovery), trust the sniff result rather than raising.
329+
if sniffed:
330+
return sniffed[0]
331+
msg = f"No vendor could be identified for file with extension '.{extension}'."
332+
raise AllotropeVendorNotFoundError(msg)
333+
334+
299335
def get_table_contents() -> str:
300336
contents = """The parsers follow maturation levels of: Recommended, Candidate Release, Working Draft.
301337

src/allotropy/parsers/agilent_gen5/agilent_gen5_parser.py

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,18 @@ class AgilentGen5Parser(VendorParser[Data, Model]):
2121
SUPPORTED_EXTENSIONS = AgilentGen5Reader.SUPPORTED_EXTENSIONS
2222
SCHEMA_MAPPER = Mapper
2323

24+
@classmethod
25+
def sniff(cls, named_file_contents: NamedFileContents) -> bool:
26+
try:
27+
raw = named_file_contents.contents.read()
28+
text = (
29+
raw.decode("utf-8", errors="replace") if isinstance(raw, bytes) else raw
30+
)
31+
lines = text.splitlines()
32+
return any(line.startswith("Software Version") for line in lines[:5])
33+
except Exception:
34+
return False
35+
2436
def create_data(self, named_file_contents: NamedFileContents) -> Data:
2537
reader = AgilentGen5Reader(named_file_contents)
2638
context = reader.extract_data_context(named_file_contents.original_file_path)

src/allotropy/parsers/agilent_gen5_image/agilent_gen5_image_parser.py

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,18 @@ class AgilentGen5ImageParser(VendorParser[Data, Model]):
2626
SUPPORTED_EXTENSIONS = AgilentGen5Reader.SUPPORTED_EXTENSIONS
2727
SCHEMA_MAPPER = Mapper
2828

29+
@classmethod
30+
def sniff(cls, named_file_contents: NamedFileContents) -> bool:
31+
try:
32+
raw = named_file_contents.contents.read()
33+
text = (
34+
raw.decode("utf-8", errors="replace") if isinstance(raw, bytes) else raw
35+
)
36+
lines = text.splitlines()
37+
return any(line.startswith("Software Version") for line in lines[:5])
38+
except Exception:
39+
return False
40+
2941
def create_data(self, named_file_contents: NamedFileContents) -> Data:
3042
reader = AgilentGen5Reader(named_file_contents)
3143

src/allotropy/parsers/agilent_openlab_cds/agilent_openlab_cds_parser.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,10 @@ class AgilentOpenLabCDSParser(VendorParser[Data, Model]):
2525
SUPPORTED_EXTENSIONS = "rslt"
2626
SCHEMA_MAPPER = Mapper
2727

28+
@classmethod
29+
def sniff(cls, _named_file_contents: NamedFileContents) -> bool:
30+
return True
31+
2832
def create_data(self, named_file_contents: NamedFileContents) -> Data:
2933
structured_data = decode_data(named_file_contents.get_bytes_stream())
3034
return Data(

src/allotropy/parsers/agilent_tapestation_analysis/agilent_tapestation_analysis_parser.py

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,19 @@ class AgilentTapestationAnalysisParser(VendorParser[Data, Model]):
2323
SUPPORTED_EXTENSIONS = "xml"
2424
SCHEMA_MAPPER = Mapper
2525

26+
@classmethod
27+
def sniff(cls, named_file_contents: NamedFileContents) -> bool:
28+
try:
29+
tree = ET.parse(named_file_contents.contents) # noqa: S314
30+
root = tree.getroot()
31+
child_tags = {
32+
child.tag.split("}")[-1] if "}" in child.tag else child.tag
33+
for child in root
34+
}
35+
return "FileInformation" in child_tags and "ScreenTapes" in child_tags
36+
except Exception:
37+
return False
38+
2639
def create_data(self, named_file_contents: NamedFileContents) -> Data:
2740
try:
2841
root_element = ET.parse( # noqa: S314

src/allotropy/parsers/appbio_absolute_q/appbio_absolute_q_parser.py

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
from __future__ import annotations
22

3+
import zipfile
4+
35
from allotropy.allotrope.models.adm.pcr.benchling._2023._09.dpcr import Model
46
from allotropy.allotrope.schema_mappers.adm.pcr.BENCHLING._2023._09.dpcr import (
57
Data,
@@ -26,6 +28,35 @@ class AppbioAbsoluteQParser(VendorParser[Data, Model]):
2628
SUPPORTED_EXTENSIONS = AppbioAbsoluteQReader.SUPPORTED_EXTENSIONS
2729
SCHEMA_MAPPER = Mapper
2830

31+
@classmethod
32+
def sniff(cls, named_file_contents: NamedFileContents) -> bool:
33+
try:
34+
if named_file_contents.extension == "zip":
35+
named_file_contents.contents.seek(0)
36+
with zipfile.ZipFile(named_file_contents.get_bytes_stream()) as zf:
37+
return any(name.endswith("_summary.csv") for name in zf.namelist())
38+
named_file_contents.contents.seek(0)
39+
raw = named_file_contents.contents.read(8192)
40+
text = (
41+
raw.decode("utf-8", errors="replace") if isinstance(raw, bytes) else raw
42+
)
43+
lines = text.splitlines()
44+
if len(lines) < 2:
45+
return False
46+
all_cols: set[str] = set()
47+
for line in lines[:2]:
48+
all_cols.update(c.strip() for c in line.split(","))
49+
has_instrument_plate = "Instrument" in all_cols and "Plate" in all_cols
50+
if has_instrument_plate:
51+
return bool(all_cols & {"Dye", "Target", "Channels"}) or any(
52+
"_target" in c for c in all_cols
53+
)
54+
return (
55+
"Run name" in all_cols and "Target" in all_cols and "Total" in all_cols
56+
)
57+
except Exception:
58+
return False
59+
2960
def create_data(self, named_file_contents: NamedFileContents) -> Data:
3061
reader = AppbioAbsoluteQReader(named_file_contents)
3162
wells = Well.create_wells(reader.data)

src/allotropy/parsers/appbio_quantstudio/appbio_quantstudio_parser.py

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
import openpyxl
2+
13
from allotropy.allotrope.models.adm.pcr.rec._2024._09.qpcr import Model
24
from allotropy.allotrope.schema_mappers.adm.pcr.rec._2024._09.qpcr import Data, Mapper
35
from allotropy.named_file_contents import NamedFileContents
@@ -31,6 +33,33 @@ class AppBioQuantStudioParser(VendorParser[Data, Model]):
3133
SUPPORTED_EXTENSIONS = AppBioQuantStudioReader.SUPPORTED_EXTENSIONS
3234
SCHEMA_MAPPER = Mapper
3335

36+
@classmethod
37+
def sniff(cls, named_file_contents: NamedFileContents) -> bool:
38+
try:
39+
if named_file_contents.extension == "txt":
40+
named_file_contents.contents.seek(0)
41+
raw = named_file_contents.contents.read(8192)
42+
text = (
43+
raw.decode("utf-8", errors="replace")
44+
if isinstance(raw, bytes)
45+
else raw
46+
)
47+
lines = text.splitlines()
48+
has_star_keys = any(
49+
line.startswith("* ") and " = " in line for line in lines[:10]
50+
)
51+
has_brackets = any(line.startswith("[") for line in lines)
52+
return has_star_keys or has_brackets
53+
# xlsx: QuantStudio XLSX files have sheet names wrapped in brackets
54+
wb = openpyxl.load_workbook(
55+
named_file_contents.get_bytes_stream(), read_only=True
56+
)
57+
sheet_names = wb.sheetnames
58+
wb.close()
59+
return any(name.startswith("[") for name in sheet_names)
60+
except Exception:
61+
return False
62+
3463
def parse_data(
3564
self, reader: AppBioQuantStudioReader, original_file_path: str
3665
) -> Data:

src/allotropy/parsers/appbio_quantstudio_designandanalysis/appbio_quantstudio_designandanalysis_parser.py

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
import openpyxl
2+
13
from allotropy.allotrope.models.adm.pcr.rec._2024._09.qpcr import Model
24
from allotropy.allotrope.schema_mappers.adm.pcr.rec._2024._09.qpcr import (
35
Data,
@@ -30,6 +32,18 @@ class AppBioQuantStudioDesignandanalysisParser(VendorParser[Data, Model]):
3032
SUPPORTED_EXTENSIONS = DesignQuantstudioReader.SUPPORTED_EXTENSIONS
3133
SCHEMA_MAPPER = Mapper
3234

35+
@classmethod
36+
def sniff(cls, named_file_contents: NamedFileContents) -> bool:
37+
try:
38+
wb = openpyxl.load_workbook(
39+
named_file_contents.get_bytes_stream(), read_only=True
40+
)
41+
sheet_names = set(wb.sheetnames)
42+
wb.close()
43+
return "Results" in sheet_names
44+
except Exception:
45+
return False
46+
3347
def parse_rt_pcr(
3448
self, reader: DesignQuantstudioReader, original_file_path: str
3549
) -> Data:

0 commit comments

Comments
 (0)