Skip to content

Commit 9e65307

Browse files
committed
feat(lance-graph-ontology): PR-bO-16 ZUGFeRD/Factur-X EN16931 via new XsdHydrator
First Tier-C hydrator (per spec) — handles XSD-shaped schemas where the OwlHydrator's TTL/RDF-XML routes don't apply. ZUGFeRD/Factur-X EN16931 is the German hybrid PDF/A-3 + XML invoice format aligned with EU directive EN 16931; the underlying XML schema is UN/CEFACT CrossIndustryInvoice (CII) v100. New XsdHydrator (minimal name-extraction shape): - crates/lance-graph-ontology/src/hydrators/xsd.rs - Walks every <xs:element>/<xs:complexType>/<xs:simpleType>/<xs:attribute>/ <xs:attributeGroup> declaration via quick-xml streaming, interns each as `{targetNamespace}#{name}` IRIs into the existing ContextBundle surface - Adds quick-xml = "0.37" as a direct dep (already transitively present via oxrdfxml) - collect_xsd_files() walks a directory tree for `.xsd`, sorted for stable interning order - Unit test verifies a tiny inline XSD interns 5 named declarations - Type-graph semantics (xs:extension base / xs:restriction base) NOT resolved into rdfs:subClassOf-equivalent edges — deferred follow-up, documented in xsd.rs module docs bO-16 ZUGFeRD (OGIT::ZUGFERD_V1, slot 30): - data/ontologies/zugferd/ — 4 XSD files (top-level CII + RAM + QDT + UDT) from the Factur-X 1.08 EN16931 profile, plus FACTUR-X_EN16931.sch Schematron rules and FACTUR-X_EN16931_codedb.xml code-list database (latter two shipped for future SchematronHydrator / CodeListHydrator PRs, not hydrated today) - Source: LandrixSoftware/validator-configuration-zugferd (Apache-2.0) - entity_count > 200 after hydration of all 4 XSDs - 17-edge cascade whitelist covers the three CII top-level relational containers (ExchangedDocument / ExchangedDocumentContext / SupplyChainTradeTransaction) plus the 14 most load-bearing RAM predicates (Seller/Buyer TradeParty, TradeAgreement / TradeDelivery / TradeSettlement headers, line-item containers, tax / payment terms, monetary summation) - 5 smoke tests verify CII root, RAM types (Header*TradeAgreementType, TradePartyType, TradeAddressType, TradeTaxType, ...), all four CII namespaces present, edge whitelist registered LICENSES/ZUGFERD.txt cites Apache-2.0 (LandrixSoftware config) + UN/CEFACT IP policy for the underlying CII XSDs. All 98 lance-graph-ontology tests pass; cargo clippy clean; downstream consumers build clean.
1 parent e041d0d commit 9e65307

16 files changed

Lines changed: 9228 additions & 5 deletions

Cargo.lock

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

LICENSES/ZUGFERD.txt

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
ZUGFeRD / Factur-X EN 16931 e-invoicing schemas
2+
================================================
3+
4+
Source
5+
------
6+
https://github.com/LandrixSoftware/validator-configuration-zugferd
7+
Upstream specification: https://www.ferd-net.de/standards/zugferd
8+
EU directive alignment: EN 16931 (semantic data model of the European
9+
core invoice).
10+
11+
The schemas are based on the UN/CEFACT Cross-Industry Invoice (CII)
12+
XML standard, profiled for the Factur-X 1.08 EN16931 conformance
13+
level — the conformance level required for B2G e-invoicing across
14+
the EU per the eInvoicing Directive 2014/55/EU.
15+
16+
Authors / publishers
17+
--------------------
18+
Validator configuration: Landrix Software (Apache-2.0).
19+
Underlying CII XSDs: UN/CEFACT (publicly available under UN/CEFACT
20+
Intellectual Property Policy — royalty-free use for implementation
21+
permitted).
22+
ZUGFeRD specification: Forum elektronische Rechnung Deutschland
23+
(FeRD).
24+
25+
License
26+
-------
27+
The validator configuration repository at
28+
LandrixSoftware/validator-configuration-zugferd is licensed under
29+
the Apache License, Version 2.0.
30+
31+
Usage in this repository
32+
------------------------
33+
data/ontologies/zugferd/ contains the four CII XSD files that make
34+
up the Factur-X 1.08 EN16931 profile:
35+
36+
FACTUR-X_EN16931.xsd (top-level)
37+
FACTUR-X_EN16931_urn_un_unece_uncefact_data_
38+
standard_ReusableAggregateBusinessInformationEntity_100.xsd (RAM types)
39+
FACTUR-X_EN16931_urn_un_unece_uncefact_data_
40+
standard_QualifiedDataType_100.xsd (QDT)
41+
FACTUR-X_EN16931_urn_un_unece_uncefact_data_
42+
standard_UnqualifiedDataType_100.xsd (UDT)
43+
44+
Plus the Schematron business-rule file `FACTUR-X_EN16931.sch` and
45+
the code-list database `FACTUR-X_EN16931_codedb.xml`. Neither is
46+
hydrated by the current XsdHydrator — they are shipped for future
47+
SchematronHydrator (business-rule projection) and CodeListHydrator
48+
(currency/unit code interning) PRs.
49+
50+
Hydration
51+
---------
52+
Hydrated by lance-graph-ontology as the L3 e-invoicing slot
53+
OGIT::ZUGFERD_V1 via XsdHydrator. The hydrator walks every
54+
`<xs:element>` / `<xs:complexType>` / `<xs:simpleType>` /
55+
`<xs:attribute>` / `<xs:attributeGroup>` declaration and interns each
56+
as `{targetNamespace}#{name}`. Type-graph semantics (`xs:extension
57+
base="..."` resolution) are NOT projected into rdfs:subClassOf
58+
edges — that's deferred to a follow-up PR.
59+
60+
Declares inherits_from: Some(OGIT::DOLCE_V1.0). Future alignment:
61+
- ram:TradePartyType ⊑ fibo-fnd:Party (FIBO Parties module)
62+
- ram:HeaderTradeSettlementHeaderMonetarySummationType.* ⊑
63+
fibo-fnd-acc-cur:MonetaryAmount
64+
- ram:TradeAddressType ⊑ schema:PostalAddress
65+
- ram:CrossIndustryInvoiceType ⊑ schema:Invoice
66+
67+
These alignments are NOT baked into the XSDs (they're XML schema
68+
constraints, not OWL axioms); they're left to downstream consumers
69+
that need the cross-G projection.

crates/lance-graph-contract/build.rs

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,10 @@ const CANONICAL_SLOTS: &[(&str, u32)] = &[
3737
// FIBO Foundations and FIBO Business Entities.
3838
("FIBOFND", 20),
3939
("FIBOBE", 21),
40+
// L3 e-invoicing schemas (PR-bO-16). ZUGFeRD/Factur-X is the
41+
// German hybrid PDF/A-3+XML invoice format aligned with EN 16931.
42+
// Hydrated as IRI-interning over UN/CEFACT CII XSDs via XsdHydrator.
43+
("ZUGFERD", 30),
4044
];
4145

4246
fn canonical_slot(token: &str) -> Option<u32> {

crates/lance-graph-ontology/Cargo.toml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,11 @@ oxrdf = "0.3"
2222
# content here.
2323
oxrdfxml = "0.2"
2424

25+
# XML parser used by XsdHydrator (ZUGFeRD/UBL/ISO 20022 ship as XSD).
26+
# Already transitively present via oxrdfxml; we declare it directly so
27+
# the XSD-name-extraction path doesn't depend on oxrdfxml's version pin.
28+
quick-xml = "0.37"
29+
2530
# Errors. thiserror keeps the public Error enum readable without serde.
2631
thiserror = "2"
2732

crates/lance-graph-ontology/src/hydrators/mod.rs

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,8 @@ pub mod provo;
2626
pub mod qudt;
2727
pub mod schemaorg;
2828
pub mod skos;
29+
pub mod xsd;
30+
pub mod zugferd;
2931

3032
pub use dolce::{hydrate_dolce, hydrate_dolce_from, hydrate_dolce_from_many};
3133
pub use fibo::{hydrate_fibo_be, hydrate_fibo_be_from, hydrate_fibo_fnd, hydrate_fibo_fnd_from};
@@ -35,3 +37,5 @@ pub use provo::{hydrate_provo, hydrate_provo_from};
3537
pub use qudt::{hydrate_qudt, hydrate_qudt_from};
3638
pub use schemaorg::{hydrate_schemaorg, hydrate_schemaorg_from};
3739
pub use skos::{hydrate_skos, hydrate_skos_from};
40+
pub use xsd::{collect_xsd_files, XsdHydrator};
41+
pub use zugferd::{hydrate_zugferd, hydrate_zugferd_from};
Lines changed: 268 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,268 @@
1+
//! Generic XSD → `ContextBundle` hydrator.
2+
//!
3+
//! Where [`super::owl::OwlHydrator`] handles OWL ontologies in Turtle and
4+
//! RDF/XML serializations, `XsdHydrator` handles XML Schema (XSD) artifacts.
5+
//! Used by ZUGFeRD / Factur-X (this PR), and intended for UBL, ISO 20022,
6+
//! and any other e-business schema delivered as XSD.
7+
//!
8+
//! Pattern D, minimal-name-extraction shape:
9+
//!
10+
//! - Walk every `<xs:element name="...">`, `<xs:complexType name="...">`,
11+
//! `<xs:simpleType name="...">`, `<xs:attribute name="...">` and
12+
//! `<xs:attributeGroup name="...">` element in each input file.
13+
//! - Compose each named declaration into a stable IRI by joining the
14+
//! nearest `xs:schema/@targetNamespace` with `#{name}` (e.g.
15+
//! `urn:un:unece:uncefact:data:standard:CrossIndustryInvoice:100#CrossIndustryInvoice`).
16+
//! - Intern each IRI exactly once into the [`super::owl::OntologySlot`]
17+
//! keyed by the chosen `G` slot.
18+
//!
19+
//! Out of scope for this PR (deferred to a full XSD type-graph projection
20+
//! follow-up):
21+
//!
22+
//! - `<xs:extension base="...">` / `<xs:restriction base="...">` resolution
23+
//! into rdfs:subClassOf-equivalent edges.
24+
//! - `<xs:element ref="...">` cross-references between files.
25+
//! - Substitution groups, attribute groups beyond name interning.
26+
//! - Schematron / business rules — those live in `.sch` files alongside the
27+
//! `.xsd` files and would be a separate Schematron hydrator.
28+
29+
use std::collections::HashMap;
30+
use std::fs;
31+
use std::path::{Path, PathBuf};
32+
use std::sync::Arc;
33+
34+
use quick_xml::events::Event;
35+
use quick_xml::Reader;
36+
37+
use super::owl::{ContextBundle, EntityId, HydrateErr, OntologySlot};
38+
use crate::registry::OntologyRegistry;
39+
40+
/// XSD hydrator. Reusable for every XSD-shaped business-document schema.
41+
pub struct XsdHydrator {
42+
pub g: u32,
43+
pub version: u32,
44+
pub domain_name: String,
45+
pub inherits_from: Option<u32>,
46+
pub starting_entity_id: EntityId,
47+
}
48+
49+
impl XsdHydrator {
50+
/// Multi-file variant. Walks every input XSD, interns named element /
51+
/// type / attribute / attributeGroup declarations as
52+
/// `{targetNamespace}#{name}` IRIs, and registers a [`ContextBundle`]
53+
/// keyed by `self.g`.
54+
pub fn hydrate_many(
55+
&self,
56+
xsd_paths: &[&Path],
57+
registry: &OntologyRegistry,
58+
) -> Result<u32, HydrateErr> {
59+
let mut iri_to_id: HashMap<String, EntityId> = HashMap::new();
60+
let mut next_id: EntityId = self.starting_entity_id;
61+
62+
let mut intern = |iri: String,
63+
map: &mut HashMap<String, EntityId>,
64+
n: &mut EntityId|
65+
-> EntityId {
66+
if let Some(&id) = map.get(&iri) {
67+
return id;
68+
}
69+
let id = *n;
70+
*n += 1;
71+
map.insert(iri, id);
72+
id
73+
};
74+
75+
for path in xsd_paths {
76+
let bytes = fs::read(path).map_err(|e| HydrateErr::Io {
77+
path: path.to_path_buf(),
78+
source: e,
79+
})?;
80+
walk_xsd(path, &bytes, &mut iri_to_id, &mut next_id, &mut intern)?;
81+
}
82+
83+
let entity_count = iri_to_id.len() as u32;
84+
let ontology = OntologySlot {
85+
entity_count,
86+
iri_to_id,
87+
};
88+
let bundle = ContextBundle {
89+
g: self.g,
90+
version: self.version,
91+
domain_name: self.domain_name.clone(),
92+
inherits_from: self.inherits_from,
93+
ontology: Some(Arc::new(ontology)),
94+
edge_types: Vec::new(),
95+
};
96+
registry.register_bundle(bundle);
97+
Ok(self.g)
98+
}
99+
}
100+
101+
/// Walk one XSD file's events, finding the schema's `targetNamespace` and
102+
/// interning every named declaration found beneath it.
103+
fn walk_xsd(
104+
path: &Path,
105+
bytes: &[u8],
106+
iri_to_id: &mut HashMap<String, EntityId>,
107+
next_id: &mut EntityId,
108+
intern: &mut dyn FnMut(String, &mut HashMap<String, EntityId>, &mut EntityId) -> EntityId,
109+
) -> Result<(), HydrateErr> {
110+
let mut reader = Reader::from_reader(bytes);
111+
reader.config_mut().trim_text(true);
112+
let mut buf = Vec::new();
113+
let mut target_ns: Option<String> = None;
114+
115+
loop {
116+
match reader.read_event_into(&mut buf) {
117+
Err(e) => {
118+
return Err(HydrateErr::Parse {
119+
path: path.to_path_buf(),
120+
message: format!("XSD: {e}"),
121+
});
122+
}
123+
Ok(Event::Eof) => break,
124+
Ok(Event::Start(e)) | Ok(Event::Empty(e)) => {
125+
// `e.name()` returns a borrowed view tied to `e`; capture
126+
// the local-name bytes into an owned slice to avoid the
127+
// temporary-borrow issue under quick-xml 0.37.
128+
let qname = e.name();
129+
let local: Vec<u8> = local_name(qname.as_ref()).to_vec();
130+
if local == b"schema" && target_ns.is_none() {
131+
target_ns = attr_value(&e, b"targetNamespace");
132+
} else if matches!(
133+
local.as_slice(),
134+
b"element" | b"complexType" | b"simpleType" | b"attribute" | b"attributeGroup"
135+
) {
136+
if let Some(name) = attr_value(&e, b"name") {
137+
let ns = target_ns.as_deref().unwrap_or("");
138+
let iri = if ns.is_empty() {
139+
// Schema with no targetNamespace: fall back to
140+
// the bare local name so the IRI is still
141+
// resolvable (used only by exotic XSDs; CII
142+
// always declares a targetNamespace).
143+
name
144+
} else {
145+
format!("{ns}#{name}")
146+
};
147+
intern(iri, iri_to_id, next_id);
148+
}
149+
}
150+
}
151+
_ => {}
152+
}
153+
buf.clear();
154+
}
155+
Ok(())
156+
}
157+
158+
fn local_name(qname: &[u8]) -> &[u8] {
159+
match qname.iter().position(|b| *b == b':') {
160+
Some(i) => &qname[i + 1..],
161+
None => qname,
162+
}
163+
}
164+
165+
fn attr_value(e: &quick_xml::events::BytesStart, attr_name: &[u8]) -> Option<String> {
166+
for attr in e.attributes().with_checks(false).flatten() {
167+
if local_name(attr.key.as_ref()) == attr_name {
168+
return std::str::from_utf8(attr.value.as_ref())
169+
.ok()
170+
.map(|s| s.to_string());
171+
}
172+
}
173+
None
174+
}
175+
176+
/// Recursively collect every `.xsd` file under `root`, sorted for stable
177+
/// IRI-interning order.
178+
pub fn collect_xsd_files(root: &Path) -> Result<Vec<PathBuf>, HydrateErr> {
179+
let mut out: Vec<PathBuf> = Vec::new();
180+
walk_dir(root, &mut out).map_err(|e| HydrateErr::Io {
181+
path: root.to_path_buf(),
182+
source: e,
183+
})?;
184+
out.sort();
185+
Ok(out)
186+
}
187+
188+
fn walk_dir(dir: &Path, out: &mut Vec<PathBuf>) -> std::io::Result<()> {
189+
if !dir.exists() {
190+
return Err(std::io::Error::new(
191+
std::io::ErrorKind::NotFound,
192+
format!("XSD directory not found: {}", dir.display()),
193+
));
194+
}
195+
if !dir.is_dir() {
196+
return Ok(());
197+
}
198+
let mut stack = vec![dir.to_path_buf()];
199+
while let Some(d) = stack.pop() {
200+
for entry in fs::read_dir(&d)? {
201+
let entry = entry?;
202+
let p = entry.path();
203+
if p.is_dir() {
204+
stack.push(p);
205+
} else if p.extension().map(|s| s == "xsd").unwrap_or(false) {
206+
out.push(p);
207+
}
208+
}
209+
}
210+
Ok(())
211+
}
212+
213+
#[cfg(test)]
214+
mod tests {
215+
use super::*;
216+
use std::io::Write;
217+
use tempfile::tempdir;
218+
219+
const TINY_XSD: &str = r#"<?xml version="1.0" encoding="UTF-8"?>
220+
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
221+
targetNamespace="urn:example:test">
222+
<xs:element name="Foo" type="xs:string"/>
223+
<xs:complexType name="BarType">
224+
<xs:sequence>
225+
<xs:element name="baz" type="xs:int"/>
226+
</xs:sequence>
227+
</xs:complexType>
228+
<xs:simpleType name="Quux"/>
229+
<xs:attribute name="anAttr" type="xs:string"/>
230+
</xs:schema>
231+
"#;
232+
233+
#[test]
234+
fn tiny_xsd_interns_named_declarations() {
235+
let dir = tempdir().unwrap();
236+
let path = dir.path().join("tiny.xsd");
237+
let mut f = std::fs::File::create(&path).unwrap();
238+
f.write_all(TINY_XSD.as_bytes()).unwrap();
239+
drop(f);
240+
241+
let reg = OntologyRegistry::new_in_memory();
242+
let h = XsdHydrator {
243+
g: 9999,
244+
version: 1,
245+
domain_name: "tiny-xsd".to_string(),
246+
inherits_from: None,
247+
starting_entity_id: 100,
248+
};
249+
h.hydrate_many(&[&path], &reg).expect("hydrate tiny XSD");
250+
251+
let bundle = reg.bundle_for(9999).expect("bundle");
252+
// Foo (element) + BarType (complexType) + baz (nested element) +
253+
// Quux (simpleType) + anAttr (attribute) = 5 names.
254+
assert!(bundle.entity_count() >= 5);
255+
assert!(bundle
256+
.resolve_iri("urn:example:test#Foo")
257+
.is_some());
258+
assert!(bundle
259+
.resolve_iri("urn:example:test#BarType")
260+
.is_some());
261+
assert!(bundle
262+
.resolve_iri("urn:example:test#baz")
263+
.is_some());
264+
assert!(bundle
265+
.resolve_iri("urn:example:test#anAttr")
266+
.is_some());
267+
}
268+
}

0 commit comments

Comments
 (0)