refactor(migration_v5): extract ontology-agnostic artifact pipeline (#172)

* refactor(migration_v5): extract ontology-agnostic artifact pipeline

The migration v5 demo's TTL → binding/DDL/property-graph
pipeline was hard-coded to MAKO. The transformations it
applied (FILL_IN resolution, cross-namespace drop, inheritance
strip) are general-purpose; only the config (TTL path,
namespace IRI, entity allowlist, annotation prefix, graph
name) is per-ontology.

## What ships

- **``ontology_artifacts.py``** — new generic module.
  ``OntologyConfig`` dataclass + ``load_ontology``,
  ``make_binding``, ``make_table_ddl``, ``make_property_graph_sql``,
  ``regenerate_snapshots`` — all parameterized by config. The
  three OWL TTL normalizations stay (they apply to any
  reasonable ontology). Annotation keys use
  ``{config.annotation_prefix}:`` instead of a hard-coded
  ``mako_demo:``.
- **``mako_artifacts.py``** — now a thin shim (~200 lines,
  down from 751). Defines ``MAKO_CONFIG`` and re-exports the
  public API (``regenerate_snapshots``, ``load_mako_ontology``,
  ``make_binding``, …) with that config bound in. The
  notebook's ``import mako_artifacts;
  mako_artifacts.regenerate_snapshots(project, dataset)``
  works unchanged.
- **``example_ontologies/simple_request_flow.ttl``** + config
  — tiny domain-neutral TTL (3 entities, 2 relationships) and
  matching ``OntologyConfig``. Proves the generic pipeline
  isn't accidentally MAKO-coupled. No runnable agent — smoke
  fixture only.
- **``tests/test_migration_v5_ontology_artifacts.py``** —
  eight tests:
  * MAKO regeneration produces byte-identical output to the
    checked-in snapshots (guards against drift).
  * Generic pipeline accepts the Simple Request Flow config
    and produces a valid binding + DDL + property graph.
  * Per-entity PK naming convention applies to non-MAKO
    configs.
  * Graph name and annotation prefix are honored per-config.
  * ``OntologyConfig`` is frozen + path properties work.

## Doc changes

- **``examples/migration_v5/README.md``** — leads with the
  pluggable contract: what ``OntologyConfig`` packages, the
  three transformations, the two example ontologies. The
  MAKO-specific design rationale stays but is reframed as
  "Reference example: MAKO design decisions". Authorship
  table updated to call out ``ontology_artifacts.py`` as the
  generic pipeline and the new ``example_ontologies/``
  directory.
- **``periodic_materialization/README.md``** — four MAKO
  references softened to "the demo's" / "the bound
  ontology" / "the migration v5 demo (MAKO is the canonical
  reference example, but the pipeline is ontology-agnostic)".

## What does NOT change

- ``mako_core.ttl`` — unchanged.
- ``mako_demo_agent.py``, ``run_agent.py``,
  ``reference_extractor.py`` — unchanged. The agent's tools
  mirror MAKO's decision flow; generalizing them would mean a
  different demo, not a generalization of this one.
- ``examples/migration_v5_demo_notebook.ipynb`` — unchanged.
  The shim preserves the notebook's import.
- Checked-in ``ontology.yaml`` / ``binding.yaml`` /
  ``table_ddl.sql`` / ``property_graph.sql`` for MAKO —
  byte-identical after the refactor (asserted by
  ``test_mako_regenerate_matches_checked_in_snapshots``).
- SDK source under ``src/`` — untouched. This is an examples
  refactor only.

## Verification

- ``PYTHONPATH=src pytest tests/test_migration_v5_ontology_artifacts.py
  tests/test_migration_v5_reference_extractor.py`` — 25/25 pass
  (8 new + 17 existing).
- ``PYTHONPATH=src python examples/migration_v5/mako_artifacts.py``
  — runs; ``git diff`` on the four MAKO snapshot files is empty.
- ``pyink --check`` + ``isort --check-only`` clean on all
  touched + new files.

* fix(migration_v5): bind declared owl:hasKey PK (PR #172 review)

Addresses two findings from the PR #172 review:

## P1 — owl:hasKey ontologies broke make_binding

The generic pipeline's first cut hard-coded ``"id"`` as the
bound primary-key property name. That worked for MAKO (where
the FILL_IN resolver synthesizes an ``id`` property on every
entity) but broke any TTL that declared ``owl:hasKey``: the
binding ended up declaring an ``id`` property the entity
didn't have, and ``load_binding_from_string`` raised
``Entity binding 'X': property 'id' is not declared on this
element``. Hard-failed the PR's "any OWL TTL" contract.

Fix in ``ontology_artifacts.py``:

* New ``_primary_key_property_name(entity)`` helper reads
  ``entity.keys.primary[0]`` (the normalization passes
  populate this for every entity — either the synthesized
  ``id`` or the TTL-declared key).
* ``make_binding`` uses it instead of ``"id"`` when emitting
  the PK property + when skipping the PK in the property
  iteration loop.
* ``make_property_graph_sql`` builds an entity → PK property
  name lookup from the ontology, then resolves the PK column
  through that name (not by matching ``p.name == "id"``).

The fix preserves MAKO byte-identical output (MAKO entities
still have ``keys.primary == ["id"]`` after FILL_IN resolution
so the lookup returns the same property).

The smoke fixture (``simple_request_flow.ttl``) didn't catch
this because it only exercised the FILL_IN path. Updated to
declare ``owl:hasKey ( rf:requestId )`` (and same for Action,
Outcome) so the existing pluggability test now genuinely
exercises the real-key path. Added two new regression tests:

* ``test_simple_request_flow_binds_declared_owl_haskey_property``
  — binding emits ``{name: requestId, column: request_id}``,
  not the synthesized ``id``.
* ``test_simple_request_flow_property_graph_resolves_owl_haskey_pk_column``
  — ``KEY (request_id)`` / ``REFERENCES request (request_id)``
  resolve through the declared PK property, not a hard-coded
  ``id``.

## P2 — README overclaimed periodic materialization pluggability

The README and playbook both implied "any OntologyConfig
works" with the periodic materialization deploy. The deploy
script actually bundles the checked-in MAKO ``binding.yaml`` /
``ontology.yaml`` / ``table_ddl.sql`` (verified in
``deploy_cloud_run_job.sh:421-423``). Running against a
different config requires regenerating those snapshots and
re-pointing the deploy at the new files — and the deploy
script doesn't yet have a CLI flag for that. Softened both
README sections to call this out explicitly; flagged the deploy
script wiring as a natural follow-up.

## INFORMATIONAL — import mode consistency in tests

The test was using package-style imports (``examples.migration_v5.X``)
that masked the fact that the shim ``mako_artifacts.py`` only
supports sibling-style imports (matching the notebook +
``run_agent.py``). Switched the test to sibling-style only so
it exercises the same import surface real callers do.

## Verification

* ``PYTHONPATH=src pytest tests/test_migration_v5_ontology_artifacts.py
  tests/test_migration_v5_reference_extractor.py`` — 27/27 pass
  (10 new + 17 existing).
* ``PYTHONPATH=src python examples/migration_v5/mako_artifacts.py``
  — outputs 18/6/7; ``git diff`` on the four MAKO snapshot
  files is empty.
* ``pyink --check`` + ``isort --check-only`` clean.

* docs(migration_v5): correct test count in README validation snippet

PR #172 follow-up review caught the validation snippet still
read ``# → 8 passed`` after the owl:hasKey regression commit
added two tests (now 10).

* fix(ci): skip ontology-artifacts tests when rdflib is absent

CI runs ``pip install -e ".[dev]"`` only — the ``[owl]`` extra
that brings in ``rdflib`` isn't installed. The new test module
pulls ``ontology_artifacts``, which imports
``bigquery_ontology.owl_importer.import_owl``, which raises
``ImportError: rdflib is required for OWL import`` during
collection. Whole module errored across Python 3.10–3.14.

Add ``pytest.importorskip("rdflib")`` at the top, mirroring
the existing pattern in ``tests/test_owl_import_bridge.py`` and
``tests/bigquery_ontology/test_owl_importer.py``. Locally
(with ``[owl]`` installed) all 10 tests still pass; on CI the
module skips cleanly instead of erroring collection.

---------

Co-authored-by: Haiyuan Cao <haiyuan@google.com>

Author: caohy1988

examples/migration_v5/README.md

@@ -0,0 +1,25 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Pluggability smoke fixtures for the migration v5 artifact pipeline.
+
+Each module in this package defines a small
+:class:`ontology_artifacts.OntologyConfig` paired with a TTL
+file. The configs aren't meant for production deploys — they
+exist to prove the pipeline in
+:mod:`ontology_artifacts` is genuinely ontology-agnostic.
+
+For the canonical reference example, see
+:mod:`mako_artifacts` in the parent package.
+"""

examples/migration_v5/example_ontologies/__init__.py

@@ -0,0 +1,125 @@
+# =============================================================================
+# Simple Request Flow — pluggability smoke fixture for the migration v5 demo
+#
+# Purpose: prove the ontology pipeline in
+#          ``examples/migration_v5/ontology_artifacts.py`` is genuinely
+#          ontology-agnostic. This TTL is intentionally domain-neutral
+#          (Request → Action → Outcome) and intentionally minimal: three
+#          entities, two relationships, no cross-namespace imports, no
+#          inheritance.
+#
+# Use:     ``regenerate_snapshots`` against
+#          :data:`example_ontologies.simple_request_flow_config.SIMPLE_REQUEST_FLOW_CONFIG`
+#          produces a valid binding + DDL + property-graph SQL with no
+#          MAKO involvement. Smoke fixture only — no runnable agent ships
+#          with this ontology.
+#
+# Why these three entities: Request is the entry point, Action is the
+# response, Outcome is the result. The minimal viable shape for a
+# directed graph that exercises ``make_binding`` (heterogeneous edges,
+# property-column generation) and ``make_property_graph_sql`` (node +
+# edge tables with KEY / SOURCE KEY / DESTINATION KEY).
+# =============================================================================
+
+@prefix rf:      <https://example.com/request-flow/> .
+@prefix owl:     <http://www.w3.org/2002/07/owl#> .
+@prefix rdf:     <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
+@prefix rdfs:    <http://www.w3.org/2000/01/rdf-schema#> .
+@prefix xsd:     <http://www.w3.org/2001/XMLSchema#> .
+@prefix dcterms: <http://purl.org/dc/terms/> .
+
+<https://example.com/request-flow/>
+    a owl:Ontology ;
+    owl:versionInfo "0.1.0" ;
+    dcterms:title "Simple Request Flow — Pluggability Smoke Fixture" ;
+    dcterms:description "Domain-neutral Request → Action → Outcome ontology used to prove the migration v5 artifact pipeline accepts any OWL TTL." ;
+    dcterms:license <https://creativecommons.org/licenses/by/4.0/> .
+
+# =============================================================================
+# Entities
+#
+# Each entity declares ``owl:hasKey`` so the pipeline exercises the real-key
+# path (not just the FILL_IN synthesis path MAKO uses). The PR review found
+# the prior version of this fixture didn't catch a bug where ``make_binding``
+# hard-coded ``id`` as the bound PK property; declaring ``owl:hasKey`` here
+# makes the regression test in ``tests/test_migration_v5_ontology_artifacts.py``
+# meaningful.
+# =============================================================================
+
+rf:Request a owl:Class ;
+    rdfs:label "Request" ;
+    rdfs:comment "An incoming ask. The entry point of the flow." ;
+    owl:hasKey ( rf:requestId ) .
+
+rf:Action a owl:Class ;
+    rdfs:label "Action" ;
+    rdfs:comment "The work performed in response to a Request." ;
+    owl:hasKey ( rf:actionId ) .
+
+rf:Outcome a owl:Class ;
+    rdfs:label "Outcome" ;
+    rdfs:comment "The result of an Action. Terminal node of the flow." ;
+    owl:hasKey ( rf:outcomeId ) .
+
+# =============================================================================
+# Data properties
+# =============================================================================
+
+rf:requestId a owl:DatatypeProperty ;
+    rdfs:label "request id" ;
+    rdfs:domain rf:Request ;
+    rdfs:range xsd:string .
+
+rf:requestText a owl:DatatypeProperty ;
+    rdfs:label "request text" ;
+    rdfs:domain rf:Request ;
+    rdfs:range xsd:string .
+
+rf:requestedAt a owl:DatatypeProperty ;
+    rdfs:label "requested at" ;
+    rdfs:domain rf:Request ;
+    rdfs:range xsd:dateTime .
+
+rf:actionId a owl:DatatypeProperty ;
+    rdfs:label "action id" ;
+    rdfs:domain rf:Action ;
+    rdfs:range xsd:string .
+
+rf:actionType a owl:DatatypeProperty ;
+    rdfs:label "action type" ;
+    rdfs:domain rf:Action ;
+    rdfs:range xsd:string .
+
+rf:executedAt a owl:DatatypeProperty ;
+    rdfs:label "executed at" ;
+    rdfs:domain rf:Action ;
+    rdfs:range xsd:dateTime .
+
+rf:outcomeId a owl:DatatypeProperty ;
+    rdfs:label "outcome id" ;
+    rdfs:domain rf:Outcome ;
+    rdfs:range xsd:string .
+
+rf:outcomeStatus a owl:DatatypeProperty ;
+    rdfs:label "outcome status" ;
+    rdfs:domain rf:Outcome ;
+    rdfs:range xsd:string .
+
+rf:completedAt a owl:DatatypeProperty ;
+    rdfs:label "completed at" ;
+    rdfs:domain rf:Outcome ;
+    rdfs:range xsd:dateTime .
+
+# =============================================================================
+# Object properties (relationships)
+# =============================================================================
+
+rf:hasAction a owl:ObjectProperty ;
+    rdfs:label "has action" ;
+    rdfs:domain rf:Request ;
+    rdfs:range rf:Action .
+
+rf:producesOutcome a owl:ObjectProperty ;
+    rdfs:label "produces outcome" ;
+    rdfs:domain rf:Action ;
+    rdfs:range rf:Outcome .

examples/migration_v5/example_ontologies/simple_request_flow.ttl

@@ -0,0 +1,45 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Pluggability smoke fixture: Simple Request Flow ontology config.
+
+Exists to prove the generic artifact pipeline in
+:mod:`ontology_artifacts` accepts any OWL TTL — not just
+MAKO's. The TTL is domain-neutral (Request → Action →
+Outcome) and intentionally simple (three entities, two
+relationships, no cross-namespace imports, no inheritance).
+
+The :data:`SIMPLE_REQUEST_FLOW_CONFIG` plugs into
+:func:`ontology_artifacts.regenerate_snapshots` exactly the
+same way :data:`mako_artifacts.MAKO_CONFIG` does. No runnable
+agent ships with this fixture — it's a smoke test for the
+pipeline, not an alternate demo.
+"""
+
+from __future__ import annotations
+
+import pathlib
+
+from ontology_artifacts import OntologyConfig
+
+_FIXTURE_DIR = pathlib.Path(__file__).parent
+
+SIMPLE_REQUEST_FLOW_CONFIG = OntologyConfig(
+    ttl_path=_FIXTURE_DIR / "simple_request_flow.ttl",
+    include_namespace="https://example.com/request-flow/",
+    entity_allowlist=("Request", "Action", "Outcome"),
+    annotation_prefix="simple_request_flow",
+    graph_name="simple_request_flow_graph",
+    snapshot_dir=_FIXTURE_DIR,
+)