Make OFN the canonical intermediate format; emit RDF/XML & OBO only at release

[matentzn]

Summary

Preliminary consensus from Slack (@balhoff, @gouttegd, @matentzn): Use OWL Functional Syntax (OFN) for all intermediate OWL artifacts in ODK builds. Convert to RDF/XML and/or OBO only as terminal release outputs.

Rationale

• RDF/XML parsing is slower; OFN speeds up builds/CI.
• OFN is easier to diff/grep during reviews.
• Cleaner, centralized prefix handling before final conversion. In fact, RDF-XML and OBO Format cannot currently handle prefixes in all cases we need them to, for example OBO Format outputs should have CURIES for exact Matches monarch-initiative/mondo#9681.

Proposal

• Update ODK Makefile so intermediate targets read/write .ofn.
• Keep .owl (RDF/XML) and .obo only in for final release targets.

[gouttegd]

What about things like import modules or components? Do you consider them as “intermediate artefacts” for the purpose of this proposal?

[matentzn]

What about things like import modules or components? Do you consider them as “intermediate artefacts” for the purpose of this proposal?

Yes but I think they are already ofn by default.. But yes, I do.

[gouttegd]

Fine by me.

I asked because the distinction between an “intermediate” artefact and a “final” artefact can sometimes be blurry. For example, the “bridge files” in Uberon are nominally only intermediate artefacts used to produce composite-metazoan.owl (the real final artefact), but they are also used by third parties (e.g. Ubergraph) – so they are intermediate artefacts from the point of view of Uberon but final artefacts from the point of view of said third parties.

[gouttegd]

Update ODK Makefile so intermediate targets read/write .ofn.

Do you want to also update the correspond filenames so that they use the .ofn extension? (E.g. rename merged_import.owl to merged_import.ofn, etc.)

Pros:

• It’s nice to have consistency between the extension of a file and its format, even though it is strictly speaking not necessary.
• It’s slightly easier when working with ROBOT because then the simple fact of passing a --output file.ofn option is enough to tell ROBOT that the output should be in OFN format – if the extension is .owl then you need an explicit convert --format ofn --output file.owl.

Cons:

• Such a switch has a potential to break a lot of things when people will update their repos.

[matentzn]

I would prefer not to update the file extensions, but could be swayed.

[gouttegd]

I would prefer not to update the file extensions

So would I (too much trouble for not enough benefits), so let’s not do it then. :)

[gouttegd]

• Update ODK Makefile so intermediate targets read/write .ofn.
• Keep .owl (RDF/XML) and .obo only in for final release targets.

Do I understand correctly that you mean that all products generated by make all_odk should be in OFN syntax only, and only converted to whatever export formats are set forth in export_formats when make prepare_release copies them over to the release directory?

[matentzn]

Do I understand correctly that you mean that all products generated by make all_odk should be in OFN syntax only, and only converted to whatever export formats are set forth in export_formats when make prepare_release copies them over to the release directory?

I would not go as far as making

src/ontology/ont.owl

(the release file created by $(ONT).owl Make target OFN just to make it RDF/XML when copied up. I think all release products should simply be RDF XML, while all edit products (imports and comonents) should be OFN by default. For me $(ONT).owl is not an intermediate - even if its rsynced to the repo root

[gouttegd]

OK, then I am not sure there is anything more to do than what I started in this draft PR.

Import modules and locally source components are already using OFN by default (more precisely they use the format specified in the global import_component_format setting, which is ofn by default). The PR ensures that OFN is also used for the merged mirror and for remotely sourced components.

What other “intermediate files” were you thinking about?

[matentzn]

What other “intermediate files” were you thinking about?

Things like EDIT_PREPROCESSED, mirrors etc

basically anything where a file ends with .owl but is not a "release file".. I can try to create an enumeration if it helped - will check out your PR later and see how complete it is

[gouttegd]

Things like EDIT_PREPROCESSED, mirrors

• EDIT_PREPROCESSED: done in the PR;
• mirrors: they already all use OFN by default, except the “merged mirror” if it is used (the PR takes care of that).

I can try to create an enumeration if it helped

That would be helpful, yes.