feat: add human_gem Python package with load_model()

data/testResults/README.md

@@ -21,11 +21,11 @@ own files. The pull request in each row is the one whose run last wrote those fi
 
 | Result file(s) | Produced by | Last updated by |
 | --- | --- | --- |
-| `qc_duplicate_keys.csv`, `qc_empty_reactions.csv`, `qc_annotation_consistency.csv`, `qc_deprecation_completeness.csv`, `qc_metabolite_completeness.csv`, `qc_reaction_sanity.csv`, `qc_duplicate_reactions.csv`, `qc_unused_entities.csv`, `qc_growth_blockers.csv` | `qcModelChecks.py` | **PR #1061** (model QC checks) |
-| `qc_annotation_issues.csv` | `annotationTest.py` | **PR #1061** (model QC checks) |
-| `qc_status.tsv` (round-trip, YAML lint, metabolic tasks, growth) | `testYamlConversion.py`, `testMetabolicTasks.py`, `action-yamllint`, `qcModelChecks.py` (via `qcStatus.py`) | **PR #1061** (model QC checks) |
-| `macaw_results.csv`, `balance_results.csv`, `qc_structure_consistency.csv` | `macawTests.py`, `balanceTest.py`, `structureConsistencyTest.py` | **PR #1061** (MACAW and balance) |
-| `memote_score.md` | `memoteSnapshot.py` (fast subset every PR; full suite via `/run memote`) | **PR #1061** (MEMOTE) |
+| `qc_duplicate_keys.csv`, `qc_empty_reactions.csv`, `qc_annotation_consistency.csv`, `qc_deprecation_completeness.csv`, `qc_metabolite_completeness.csv`, `qc_reaction_sanity.csv`, `qc_duplicate_reactions.csv`, `qc_unused_entities.csv`, `qc_growth_blockers.csv` | `qcModelChecks.py` | **PR #1069** (model QC checks) |
+| `qc_annotation_issues.csv` | `annotationTest.py` | **PR #1069** (model QC checks) |
+| `qc_status.tsv` (round-trip, YAML lint, metabolic tasks, growth) | `testYamlConversion.py`, `testMetabolicTasks.py`, `action-yamllint`, `qcModelChecks.py` (via `qcStatus.py`) | **PR #1069** (model QC checks) |
+| `macaw_results.csv`, `balance_results.csv`, `qc_structure_consistency.csv` | `macawTests.py`, `balanceTest.py`, `structureConsistencyTest.py` | **PR #1069** (MACAW and balance) |
+| `memote_score.md` | `memoteSnapshot.py` (fast subset every PR; full suite via `/run memote`) | **PR #1069** (MEMOTE) |
 | `gene-essential.csv`, `gene-essential_summary.md` | `geneEssentiality.py` via `/run gene-essentiality` | **PR #1027** (gene essentiality) |
 
 ## 2. What each check means

data/testResults/model_qc_summary.md

@@ -1,59 +1,59 @@
 ## Model quality report
 
-:warning: **6 pre-existing finding(s), no regressions vs `main`.** Non-blocking.
+:warning: **6 pre-existing finding(s), no regressions vs `develop`.** Non-blocking.
 
-_Each check name links to its explanation in the [testResults README](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/README.md)._
+_Each check name links to its explanation in the [testResults README](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/README.md)._
 
 ### Model checks
 _Duplicate keys (model unloadable) and no growth block the merge; every other row is a non-blocking report._
 
-| Check | Result | Δ vs `main` | |
+| Check | Result | Δ vs `develop` | |
 | --- | ---: | ---: | :---: |
-| [Duplicate `!!omap` keys](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/README.md#duplicate-omap-keys) | 0 | new | :white_check_mark: |
-| [Growth (biomass producible)](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/README.md#growth-biomass-producible) | 125 | new | :white_check_mark: |
-| [Reactions with no metabolites](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/README.md#reactions-with-no-metabolites) | 0 | new | :white_check_mark: |
-| [Model / annotation-table inconsistencies](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/README.md#model--annotation-table-inconsistencies) | 0 | new | :white_check_mark: |
-| [Removed reactions or metabolites not deprecated](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/README.md#removed-reactions-or-metabolites-not-deprecated) | 0 | new | :white_check_mark: |
-| [Metabolites missing formula](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/README.md#metabolites-missing-formula) | 0 | new | :white_check_mark: |
-| [Metabolites missing charge](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/README.md#metabolites-missing-charge) | 0 | new | :white_check_mark: |
-| [Reaction bound / GPR issues](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/README.md#reaction-bound--gpr-issues) | 0 | new | :white_check_mark: |
-| [Exact-duplicate reaction groups](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/README.md#exact-duplicate-reaction-groups) | 0 | new | :white_check_mark: |
-| [Unused metabolites](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/README.md#unused-metabolites) | 0 | new | :white_check_mark: |
-| [Unused genes](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/README.md#unused-genes) | 0 | new | :white_check_mark: |
-| [Malformed cross-references](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/README.md#malformed-cross-references) | 0 | new | :white_check_mark: |
-| [Cross-refs inconsistent across compartments](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/README.md#cross-refs-inconsistent-across-compartments) | [3](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/qc_annotation_issues.csv) | new | :warning: |
+| [Duplicate `!!omap` keys](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/README.md#duplicate-omap-keys) | 0 | 0 | :white_check_mark: |
+| [Growth (biomass producible)](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/README.md#growth-biomass-producible) | 125 | 0 | :white_check_mark: |
+| [Reactions with no metabolites](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/README.md#reactions-with-no-metabolites) | 0 | 0 | :white_check_mark: |
+| [Model / annotation-table inconsistencies](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/README.md#model--annotation-table-inconsistencies) | 0 | 0 | :white_check_mark: |
+| [Removed reactions or metabolites not deprecated](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/README.md#removed-reactions-or-metabolites-not-deprecated) | 0 | 0 | :white_check_mark: |
+| [Metabolites missing formula](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/README.md#metabolites-missing-formula) | 0 | 0 | :white_check_mark: |
+| [Metabolites missing charge](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/README.md#metabolites-missing-charge) | 0 | 0 | :white_check_mark: |
+| [Reaction bound / GPR issues](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/README.md#reaction-bound--gpr-issues) | 0 | 0 | :white_check_mark: |
+| [Exact-duplicate reaction groups](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/README.md#exact-duplicate-reaction-groups) | 0 | 0 | :white_check_mark: |
+| [Unused metabolites](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/README.md#unused-metabolites) | 0 | 0 | :white_check_mark: |
+| [Unused genes](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/README.md#unused-genes) | 0 | 0 | :white_check_mark: |
+| [Malformed cross-references](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/README.md#malformed-cross-references) | 0 | 0 | :white_check_mark: |
+| [Cross-refs inconsistent across compartments](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/README.md#cross-refs-inconsistent-across-compartments) | [3](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/qc_annotation_issues.csv) | 0 | :warning: |
 
 ### MACAW and mass/charge balance
 
-| Check | Result | Δ vs `main` | |
+| Check | Result | Δ vs `develop` | |
 | --- | ---: | ---: | :---: |
-| [Reactions flagged by MACAW dead-end test](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/README.md#reactions-flagged-by-macaw-dead-end-test) | [2510](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/macaw_results.csv) | -703 | :warning: |
-| [Reactions flagged as MACAW duplicates](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/README.md#reactions-flagged-as-macaw-duplicates) | [377](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/macaw_results.csv) | -2 | :warning: |
-| [Mass-imbalanced reactions](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/README.md#mass-imbalanced-reactions) | [87](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/balance_results.csv) | new | :warning: |
-| [Charge-imbalanced reactions](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/README.md#charge-imbalanced-reactions) | [234](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/balance_results.csv) | new | :warning: |
-| [Structure vs formula/charge inconsistencies](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/README.md#structure-vs-formulacharge-inconsistencies) | [397](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/qc_structure_consistency.csv) | new | :warning: |
+| [Reactions flagged by MACAW dead-end test](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/README.md#reactions-flagged-by-macaw-dead-end-test) | [2510](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/macaw_results.csv) | 0 | :warning: |
+| [Reactions flagged as MACAW duplicates](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/README.md#reactions-flagged-as-macaw-duplicates) | [377](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/macaw_results.csv) | 0 | :warning: |
+| [Mass-imbalanced reactions](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/README.md#mass-imbalanced-reactions) | [87](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/balance_results.csv) | 0 | :warning: |
+| [Charge-imbalanced reactions](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/README.md#charge-imbalanced-reactions) | [234](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/balance_results.csv) | 0 | :warning: |
+| [Structure vs formula/charge inconsistencies](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/README.md#structure-vs-formulacharge-inconsistencies) | [397](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/qc_structure_consistency.csv) | 0 | :warning: |
 
 ### Model file and metabolic tasks
 
 | Check | Result | |
 | --- | ---: | :---: |
-| [YAML round-trip (cobrapy)](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/README.md#yaml-round-trip-cobrapy) | pass | :white_check_mark: |
-| [YAML round-trip (RAVEN)](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/README.md#yaml-round-trip-raven) | pass | :white_check_mark: |
-| [YAML lint](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/README.md#yaml-lint) | pass | :white_check_mark: |
-| [Essential metabolic tasks](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/README.md#essential-metabolic-tasks) | 57 passed | :white_check_mark: |
-| [Verification metabolic tasks](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/README.md#verification-metabolic-tasks) | 21 passed | :white_check_mark: |
+| [YAML round-trip (cobrapy)](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/README.md#yaml-round-trip-cobrapy) | pass | :white_check_mark: |
+| [YAML round-trip (RAVEN)](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/README.md#yaml-round-trip-raven) | pass | :white_check_mark: |
+| [YAML lint](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/README.md#yaml-lint) | pass | :white_check_mark: |
+| [Essential metabolic tasks](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/README.md#essential-metabolic-tasks) | 57 passed | :white_check_mark: |
+| [Verification metabolic tasks](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/README.md#verification-metabolic-tasks) | 21 passed | :white_check_mark: |
 
-### [MEMOTE](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/README.md#memote)
+### [MEMOTE](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/README.md#memote)
 
-**Total score: 63.2%** (core subset)  
+**Total score: 63.2%** (core subset)   0
 
 | Section | Score | Δ vs base |
 | --- | ---: | ---: |
-| consistency | 42.4% |  |
-| annotation_met | 73.0% |  |
-| annotation_rxn | 72.7% |  |
-| annotation_gene | 46.7% |  |
-| annotation_sbo | 81.7% |  |
+| consistency | 42.4% | 0 |
+| annotation_met | 73.0% | 0 |
+| annotation_rxn | 72.7% | 0 |
+| annotation_gene | 46.7% | 0 |
+| annotation_sbo | 81.7% | 0 |
 
 <details><summary>Per-test scores</summary>
 
@@ -89,11 +89,11 @@ _Duplicate keys (model unloadable) and no growth block the merge; every other ro
 
 </details>
 
-**Full suite: 64.2%** &nbsp;  &middot; _from the last_ `/run memote`.
+**Full suite: 64.2%** &nbsp; 0 &middot; _from the last_ `/run memote`.
 
 _The score above is the fast core subset. Comment_ `/run memote` _to run the full suite on this pull request; the score updates here when it finishes._
 
-### [Gene essentiality (Hart 2015)](https://github.com/SysBioChalmers/Human-GEM/blob/develop/data/testResults/README.md#gene-essentiality-hart-2015)
+### [Gene essentiality (Hart 2015)](https://github.com/SysBioChalmers/Human-GEM/blob/feat/human-gem-python-package/data/testResults/README.md#gene-essentiality-hart-2015)
 
 _Not run automatically (it takes hours). Comment_ `/run gene-essentiality` _to run it on this pull request; the result posts as its own comment._
 

human_gem/__init__.py

@@ -0,0 +1,18 @@
+"""Python interface to the Human-GEM genome-scale metabolic model.
+
+Loads Human-GEM as a cobrapy model with its full cross-reference and SBO
+annotations, on top of `raven-toolbox`. The heavy machinery (YAML I/O, ftINIT,
+metabolic tasks, gap-filling) lives in raven-toolbox; this package adds the
+Human-GEM-specific glue - most importantly loading the model with the
+annotation tables (`reactions.tsv` / `metabolites.tsv` / `genes.tsv`) merged
+onto it, which a bare `cobra.io.load_yaml_model` does not do.
+
+    import human_gem
+    model = human_gem.load_model()          # annotated cobra.Model
+"""
+from __future__ import annotations
+
+from .io import load_model
+
+__version__ = "0.1.0"
+__all__ = ["load_model"]

human_gem/annotation.py

@@ -0,0 +1,130 @@
+"""Merge the annotation tables (cross-references + SBO) onto a Human-GEM model.
+
+The YAML model stores only inline fields (`eccodes`, `metFrom`, `smiles`); the
+external identifiers live in `reactions.tsv` / `metabolites.tsv` / `genes.tsv`.
+This reads those tables and writes the ids onto each cobra entity's
+`annotation` dict (namespace -> list of ids), then adds SBO terms via
+raven-toolbox's canonical `add_sbo_terms`.
+
+This is the packaged home of the logic that also lives in `code/annotateGEM.py`
+(the CI helper); that module is intended to become a thin shim importing from
+here so there is a single source of truth.
+"""
+from __future__ import annotations
+
+from pathlib import Path
+
+import cobra
+import pandas as pd
+from raven_toolbox.annotation import add_sbo_terms
+
+# TSV column -> identifiers.org namespace (from annotateGEM.m id2miriam).
+RXN_ID2MIRIAM = {
+    "rxnKEGGID": "kegg.reaction",
+    "rxnBiGGID": "bigg.reaction",
+    "rxnREACTOMEID": "reactome",
+    "rxnRecon3DID": "vmhreaction",
+    "rxnMetaNetXID": "metanetx.reaction",
+    "rxnTCDBID": "tcdb",
+    "rxnRheaID": "rhea",
+    "rxnRheaMasterID": "rhea",
+}
+MET_ID2MIRIAM = {
+    "metBiGGID": "bigg.metabolite",
+    "metKEGGID": "kegg.compound",
+    "metHMDBID": "hmdb",
+    "metChEBIID": "chebi",
+    "metPubChemID": "pubchem.compound",
+    "metLipidMapsID": "lipidmaps",
+    "metRecon3DID": "vmhmetabolite",
+    "metMetaNetXID": "metanetx.chemical",
+    "metSeedID": "seed.compound",
+}
+GENE_ID2MIRIAM = {
+    "genes": "ensembl",
+    "geneENSTID": "ensembl",
+    "geneENSPID": "ensembl",
+    "geneUniProtID": "uniprot",
+    "geneSymbols": "hgnc.symbol",
+    "geneEntrezID": "ncbigene",
+}
+
+_SBO_GENE = "SBO:0000243"  # gene; add_sbo_terms covers reactions/metabolites, not genes
+_BIOMASS_RXN_NAME = "Generic human cell biomass reaction"
+
+
+def _read_tsv(path: Path) -> pd.DataFrame:
+    """Read a TSV annotation table as text (empty cells become ``""``)."""
+    return pd.read_csv(path, sep="\t", dtype=str, keep_default_na=False)
+
+
+def _split_ids(cell: str) -> list[str]:
+    """Split a ``";"``-separated annotation cell into clean, non-empty ids."""
+    return [part.strip() for part in str(cell).split(";") if part.strip()]
+
+
+def _chebi(ids: list[str]) -> list[str]:
+    """Ensure every ChEBI id carries the ``CHEBI:`` prefix."""
+    return [i if i.upper().startswith("CHEBI:") else f"CHEBI:{i}" for i in ids]
+
+
+def _rhea(ids: list[str]) -> list[str]:
+    """Strip the ``RHEA:`` prefix (it must not appear in the identifiers.org URL)."""
+    return [i[5:] if i.upper().startswith("RHEA:") else i for i in ids]
+
+
+def _apply_row(annotation: dict, row: pd.Series, id2miriam: dict) -> None:
+    """Add the mapped id columns of ``row`` to ``annotation`` (namespace -> list)."""
+    for column, namespace in id2miriam.items():
+        if column not in row:
+            continue
+        ids = _split_ids(row[column])
+        if namespace == "chebi":
+            ids = _chebi(ids)
+        elif namespace == "rhea":
+            ids = _rhea(ids)
+        if not ids:
+            continue
+        merged = list(annotation.get(namespace, []))
+        merged.extend(ids)
+        # Dedupe, preserving first-seen order (columns can share a namespace).
+        annotation[namespace] = list(dict.fromkeys(merged))
+
+
+def annotate_gem(
+    model: cobra.Model,
+    model_dir: str | Path,
+    *,
+    types: tuple[str, ...] = ("rxn", "met", "gene"),
+) -> cobra.Model:
+    """Merge the TSV cross-references and SBO terms into ``model`` in place.
+
+    Returns the same ``model`` object (pass a copy to keep an un-annotated one).
+    """
+    model_dir = Path(model_dir)
+
+    if "met" in types:
+        mets = _read_tsv(model_dir / "metabolites.tsv").set_index("mets")
+        for met in model.metabolites:
+            if met.id in mets.index:
+                _apply_row(met.annotation, mets.loc[met.id], MET_ID2MIRIAM)
+
+    if "rxn" in types:
+        rxns = _read_tsv(model_dir / "reactions.tsv").set_index("rxns")
+        for rxn in model.reactions:
+            if rxn.id in rxns.index:
+                _apply_row(rxn.annotation, rxns.loc[rxn.id], RXN_ID2MIRIAM)
+
+    if "gene" in types:
+        genes = _read_tsv(model_dir / "genes.tsv").set_index("genes")
+        for gene in model.genes:
+            if gene.id in genes.index:
+                row = genes.loc[gene.id].copy()
+                row["genes"] = gene.id  # the gene id itself is an ensembl id
+                _apply_row(gene.annotation, row, GENE_ID2MIRIAM)
+            gene.annotation["sbo"] = _SBO_GENE
+
+    if "met" in types or "rxn" in types:
+        add_sbo_terms(model, biomass_rxn_name=_BIOMASS_RXN_NAME)
+
+    return model