metanorma site generate ignore errors collected on compile

[CAMOBAP]

Intro

During investigation of mn2pdf fatal error I have noted that metanorma finishes successfully in case of Fatal: ... errors

Problem

Compiler::compile_file returns an array of fatal errors which is ignored:

metanorma-cli/lib/metanorma/cli/site_generator.rb

Line 31 in 7663824

select_source_files.each { |source| compile(source) }

Proposed solution

• exit code must not be zero if we faced a fatal error
• print fatal error by the end of site generation

@opoudjis @ronaldtse I would like to get your feedback on proposed solution

[CAMOBAP]

Examples of errors that do not affect exit codes:

• https://github.com/metanorma/metanorma-snap/actions/runs/4383904075/jobs/7676930748#step:6:115
• https://github.com/metanorma/metanorma-snap/actions/runs/4383904075/jobs/7676895856#step:6:36
• https://github.com/metanorma/metanorma-cli/actions/runs/4403928757/jobs/7714663157#step:10:148

[opoudjis]

I'm afraid metanorma-cli falls in between areas of responsibility: I didn't author it, and I haven't particularly been maintaining it. I agree that fatals should be propagated up; wouldn't the simplest thing to do be to raise an error if a fatal error is returned?

(I suppress errors in metanorma gems so that the file can be processed to completion, with a complete list of errors, and a complete *.xml.abort file generated for debugging. But nothing prevents an error being re-raised by whatever is consuming the list of errors.)

I am vague as to whether the @fatalerror variable in metanorma-standoc is what is going into the compile.error in CLI. I don't think they are: the errors in metanorma gem lib/metanorma/compile.rb seem to be populated only from isodoc crashes. So when process_input_adoc() in that file processes the Asciidoctor (as inherited from metanorma-standoc). There is an attempt to do so in lib/metanorma/input/asciidoc.rb in asciidoctor_validate(), grabbing content from STDERR so long as it has the right prefix; but I'm pretty sure it's ignoring @fatalerror.

(OTOH, if fatalerror is triggered in metanorma-standoc, then no Semantic XML file will be generated, and that alone should be enough to abort execution, because the isodoc conversion has nothing to proceed from.)

I'm afraid I do not and will not have the free time to fix this myself, but that at least gives you some pointers. If errors can be trapped in isodoc and standoc, and then propagated back up through well-managed channels to metanorma gem and metanorma-cli, that is a good thing, but as you can see, that propagation until now has been piecemeal and haphazard.

[CAMOBAP]

I'm afraid metanorma-cli falls in between areas of responsibility: I didn't author it, and I haven't particularly been maintaining it. I agree that fatals should be propagated up; wouldn't the simplest thing to do be to raise an error if a fatal error is returned?

I think we already have a strict node for this

But the original problem is that metanorma returns zero exit code in that case which is definitely wrong

To be on the same page I don't propose any massive refactoring/changes, just two simple things:

• non-zero exit code
• print fatal error to the stdout

[opoudjis]

FWIW, I'm still in agreement with this; should I assign you the task, @CAMOBAP ?

[ronaldtse]

@CAMOBAP how's the progress of this task? Thanks.

[ronaldtse]

@CAMOBAP is working on this. Thanks!