Refactor: autoload, OOP cleanup, and comprehensive spec improvements

- Convert all require/require_relative to Ruby autoload
- Extract Collection.parse_formats, ConfigExporter
- Add SiteManifest models using lutaml-model
- Improve Compiler: private validate_file_path, simplify options serialization
- Clean up FatalCompilationError: pass message to super
- Catch SystemExit from upstream gems, convert to FatalCompilationError
- Remove all doubles/mocks from specs, use real instances
- Fix font-related test failures with continue_without_fonts
- Use portable output formats for IETF/ITU fixtures (no xml2rfc/PDF dependency)
- Create real template/stylesheet files in tests (previously masked by mocks)
- Add collection rendering and config export specs

Author: ronaldtse

.rspec

@@ -1,3 +1,3 @@
 --color
---profile
+--format progress
 --require spec_helper

CLAUDE.md

@@ -0,0 +1,97 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+metanorma-cli is the command-line interface for the Metanorma document processing toolchain. It provides the `metanorma` executable for compiling Metanorma documents across multiple standard flavors (ISO, IEC, IETF, BIPM, ITU, OGC, IEEE, JIS, etc.). It wraps the `metanorma` gem and adds CLI-level features: document compilation, collection rendering, site generation, template management, and STS XML conversion via `mnconvert`.
+
+## Build & Development Commands
+
+```bash
+# Install dependencies
+bundle install
+
+# Run full test suite
+bundle exec rake
+
+# Run a single spec file
+bundle exec rspec spec/metanorma/cli/compiler_spec.rb
+
+# Run a single example by line number
+bundle exec rspec spec/metanorma/cli/compiler_spec.rb:42
+
+# Lint
+bundle exec rubocop
+
+# Run the CLI locally
+bundle exec exe/metanorma compile document.adoc -t iso
+
+# IRB console with project loaded
+bin/console
+```
+
+The project uses direnv + nix flake (see `flake.nix`) for environment setup. If using nix, run `direnv allow` to enter the dev shell.
+
+Tests use RSpec with VCR cassettes (for HTTP stubbing), webmock, and rspec-command. SimpleCov is configured for coverage. Test state persistence is in `.rspec_status`.
+
+## Architecture
+
+### Entry Points
+
+- **`exe/metanorma`** — Main CLI executable. Sets up encoding, handles SOCKS proxy support via `socksify`, then delegates to `Metanorma::Cli.start(ARGV)`.
+- **`exe/metanorma-manifest`** — Standalone tool that converts YAML manifests into Metanorma XML/HTML collection indexes.
+- **`lib/metanorma-cli.rb`** — Just requires `metanorma/cli`.
+- **`lib/metanorma/cli.rb`** — Core module. `Cli.start` intercepts arguments: if no known command is found, it prepends `"compile"` (so bare `metanorma foo.adoc` works). Defines config paths (`~/.metanorma/config.yml` global, `.metanorma/config.yml` local).
+
+### Command System (Thor)
+
+- **`Command`** (`lib/metanorma/cli/command.rb`) — Main Thor command class. Defines all top-level commands: `compile`, `collection`, `convert`, `version`, `new`, `log_messages`, `list-extensions`, `list-doctypes`, `export-config`. Also has three subcommands: `template-repo`, `site`, `config`.
+- **`ThorWithConfig`** (`lib/metanorma/cli/thor_with_config.rb`) — Base class for commands. Overrides `options` to merge CLI arguments with values from global and local config files (`~/.metanorma/config.yml` and `.metanorma/config.yml`). The merging is done by `Commands::Config.load_configs`.
+- **`flavor.rb`** (module, not the one in `lib/metanorma/`) — Mixed into `Command`. Provides flavor discovery, extension listing, doctype tables, backend version lookups, and the `export-config` command.
+
+### Key Classes
+
+- **`Compiler`** (`compiler.rb`) — Wraps `Metanorma::Compile`. Validates input file existence, normalizes options (splits extract/extensions on commas), and calls the metanorma core compile pipeline.
+- **`Collection`** (`collection.rb`) — Renders Metanorma collections from YAML or XML files. Delegates to `Metanorma::Collection.parse` for the actual processing.
+- **`SiteGenerator`** (`site_generator.rb`) — Generates an HTML site from a collection of Metanorma documents. Reads a `metanorma.yml` manifest (via `SiteManifest` models), compiles individual `.adoc` files and collection files, builds a Relaton collection index, then converts to HTML using Liquid templates.
+- **`Generator`** (`generator.rb`) — Creates new Metanorma documents from templates. Merges base templates (`templates/base/`) with type-specific templates downloaded from Git repos.
+- **`GitTemplate`** (`git_template.rb`) — Manages Git-hosted document templates. Clones template repos to `~/.metanorma/templates/git/`.
+- **`TemplateRepo`** (`template_repo.rb`) — Manages template repo registrations in the global config file.
+
+### Subcommands
+
+- **`Commands::Config`** (`commands/config.rb`) — `metanorma config get/set/unset`. Manages hierarchical YAML config (global + local). The `load_configs` class method merges global then local config into CLI options.
+- **`Commands::Site`** (`commands/site.rb`) — `metanorma site generate`. Resolves manifest paths, stylesheet/template paths, and delegates to `SiteGenerator`.
+- **`Commands::TemplateRepo`** (`commands/template_repo.rb`) — `metanorma template-repo add`. Registers template repos.
+
+### Flavor System
+
+- **`Metanorma::Flavor`** (`lib/metanorma/flavor.rb`) — Activates and loads all supported metanorma flavor gems (`metanorma-iso`, `metanorma-iec`, etc.). Activation happens at require time (line 77). Private gems (nist, ribose) fail silently.
+- **`Metanorma::SiteManifest`** (`lib/metanorma/site_manifest.rb`) — Lutaml::Model serializable classes for parsing `metanorma.yml` site manifest files.
+
+### Supporting Code
+
+- **`UI`** (`ui.rb`) — Thin Thor-based wrapper for console output (say, info, error, table).
+- **`Errors`** (`errors.rb`) — Custom error classes: `FileNotFoundError`, `FatalCompilationError`, `InvalidManifestFileError`, `DuplicateTemplateError`.
+- **`stringify_all_keys.rb`** — Core extensions for deep key stringification/symbolization on Hash and Array. Note: `metanorma-utils` provides similar functionality via `Metanorma::Utils::Hash`.
+
+## Configuration
+
+Config files are YAML with a `cli` key at the top level. Values from config are merged into Thor options (global first, then local). Example:
+
+```yaml
+cli:
+  agree_to_terms: true
+  install_fonts: false
+```
+
+- Global: `~/.metanorma/config.yml`
+- Local: `.metanorma/config.yml`
+
+## Testing Notes
+
+- Spec helper mocks `Mn2pdf.convert` and `MnConvert.convert` to copy files instead of running real conversions.
+- `strip_guid` helper normalizes UUIDs in XML output for deterministic comparisons.
+- Acceptance specs (`spec/acceptance/`) test the CLI end-to-end via `rspec-command`.
+- VCR cassettes are in `spec/vcr_cassettes/` for HTTP request stubs.

lib/metanorma/cli.rb

@@ -2,31 +2,32 @@
 
 require "metanorma"
 require "metanorma/flavor"
-
 require "metanorma/site_manifest"
 
-require "metanorma/cli/version"
-require "metanorma/cli/errors"
-require "metanorma/cli/command"
-
 module Metanorma
   module Cli
+    autoload :Collection, "metanorma/cli/collection"
+    autoload :Command, "metanorma/cli/command"
+    autoload :Commands, "metanorma/cli/commands"
+    autoload :Compiler, "metanorma/cli/compiler"
+    autoload :ConfigExporter, "metanorma/cli/config_exporter"
+    autoload :Errors, "metanorma/cli/errors"
+    autoload :FlavorMethods, "metanorma/cli/flavor"
+    autoload :Generator, "metanorma/cli/generator"
+    autoload :GitTemplate, "metanorma/cli/git_template"
+    autoload :SiteGenerator, "metanorma/cli/site_generator"
+    autoload :TemplateRepo, "metanorma/cli/template_repo"
+    autoload :ThorWithConfig, "metanorma/cli/thor_with_config"
+    autoload :UI, "metanorma/cli/ui"
+    autoload :VERSION, "metanorma/cli/version"
+
     CONFIG_DIRNAME = ".metanorma"
     CONFIG_FILENAME = "config.yml"
 
     def self.load_flavors
       Metanorma::Flavor.load_flavors
     end
 
-    # Invoking commands
-    #
-    # In the Metanorma CLI, we've included some custom behavior,
-    # like exposing the compiation directly from the root command.
-    #
-    # So, for this use case we first check if the user is actually
-    # trying to compile a document or not, and based on that we'll
-    # compile the document or show the help documentation.
-    #
     def self.start(arguments)
       if find_command(arguments).empty?
         arguments.unshift("compile")

lib/metanorma/cli/collection.rb

@@ -16,6 +16,11 @@ def self.render(filename, options = {})
         new(filename, options).render
       end
 
+      def self.parse_formats(formats)
+        formats = formats.split(",") if formats.is_a?(String)
+        (formats || []).map { |extension| extension.strip.to_sym }
+      end
+
       def render
         extract_options_from_file
         collection_file.render(collection_options.compact)
@@ -37,7 +42,7 @@ def collection_options
           compile: @compile_options,
           output_folder: build_output_folder,
           coverpage: options.fetch(:coverpage, nil),
-          format: collection_output_formats(options.fetch(:format, "")),
+          format: self.class.parse_formats(options.fetch(:format, "")),
           site_generate: options["site_generate"],
         }
       end
@@ -54,14 +59,6 @@ def build_output_folder
         end
       end
 
-      def collection_output_formats(formats)
-        if formats.is_a?(String)
-          formats = formats.split(",")
-        end
-
-        (formats || []).map { |extension| extension.strip.to_sym }
-      end
-
       def extract_options_from_file
         yaml_file = if /\.ya?ml$/.match?(file.to_s)
                       YAML.safe_load(File.read(file.to_s))

lib/metanorma/cli/command.rb

@@ -1,19 +1,12 @@
 # frozen_string_literal: true
 
-require "metanorma/cli/compiler"
-require "metanorma/cli/generator"
-require "metanorma/cli/collection"
-require "metanorma/cli/git_template"
-require "metanorma/cli/thor_with_config"
-require "metanorma/cli/commands/config"
-require "metanorma/cli/commands/template_repo"
-require "metanorma/cli/commands/site"
 require "mnconvert"
-require_relative "flavor"
 
 module Metanorma
   module Cli
     class Command < ThorWithConfig
+      include FlavorMethods
+
       class_option :progress, aliases: "-s", type: :boolean, default: false,
                               desc: "Show progress for long running tasks" \
                                     " (like download)"
@@ -193,7 +186,7 @@ def list_doctypes(type = nil)
 
       desc "export-config", "Export flvor configuration"
       def export_config(type = nil)
-        export_config_flavor(type)
+        ConfigExporter.new(type).export
       end
 
       desc "template-repo", "Manage metanorma templates repository"

lib/metanorma/cli/commands.rb

@@ -0,0 +1,9 @@
+module Metanorma
+  module Cli
+    module Commands
+      autoload :Config, "metanorma/cli/commands/config"
+      autoload :Site, "metanorma/cli/commands/site"
+      autoload :TemplateRepo, "metanorma/cli/commands/template_repo"
+    end
+  end
+end

lib/metanorma/cli/commands/config.rb

@@ -1,8 +1,6 @@
 require "pathname"
 require "metanorma-utils"
 
-# require "metanorma/cli/stringify_all_keys"
-
 module Metanorma
   module Cli
     module Commands
@@ -48,17 +46,6 @@ def unset(name)
 
         def self.exit_on_failure?() true end
 
-        # priority:
-        # IDEAL:
-        # thor defaults -> global conf -> local conf
-        #   -> env vars -> passed arguments
-        #
-        # ACTUAL:
-        # all arguments -> global conf -> local conf
-        #
-        # - thor doesn't provide to differentiate default values against passed
-        #   args
-        # - thor doesn't allow to get all args available for current command
         def self.load_configs(options, configs = [
           Metanorma::Cli.global_config_path, Metanorma::Cli.local_config_path
         ])
@@ -108,19 +95,15 @@ def dig_path(str)
         end
 
         def deep_set(hash, value, *keys)
-          keys[0...-1].reduce(hash) do |acc, h|
-            tmp = acc.public_send(:[], h)
-            if tmp.nil?
-              acc[h] = tmp = Hash.new
-            end
-            tmp
-          end.public_send(:[]=, keys.last, value)
+          leaf = keys[0...-1].reduce(hash) do |acc, k|
+            acc[k] ||= {}
+          end
+          leaf[keys.last] = value
         end
 
         def deep_unset(hash, *keys)
-          keys[0...-1].reduce(hash) do |acc, h|
-            acc.public_send(:[], h)
-          end.delete(keys.last)
+          leaf = keys[0...-1].reduce(hash) { |acc, k| acc[k] }
+          leaf.delete(keys.last)
         end
 
         def try_convert_to_bool(value)

lib/metanorma/cli/commands/site.rb

@@ -1,8 +1,6 @@
 # frozen_string_literal: true
 
 require "pathname"
-require "metanorma/cli/site_generator"
-require "metanorma/cli/thor_with_config"
 
 module Metanorma
   module Cli
@@ -58,28 +56,6 @@ class Site < ThorWithConfig
                type: :boolean,
                desc: "Strict compilation: abort if there are any errors"
 
-        # If no argument is provided, work out the base
-        # path to use for calculation of full paths for
-        # files referenced in the site manifest.
-        #
-        # Additionally, if the config file is not provided,
-        # use the current working directory as the base path.
-        # If the config file is provided, use the directory
-        # of the config file as the base path.
-        #
-        # If the source path is a file and no config file is provided,
-        # treat the source path as the config file.
-        # Similar to the above, use the directory of the config file
-        # as the base path.
-        #
-        # For stylesheet and template_dir options, and if they are
-        # relative paths, they will be resolved relative to whatever
-        # defined them.
-        #
-        # So, if they are provided via the command line,
-        # resolve them relative to the current working directory.
-        # If they are provided via the site manifest,
-        # resolve them relative to the site manifest's directory.
         def generate(manifest_path = nil)
           my_options = options.dup # because options is not modifiable
 
@@ -116,21 +92,14 @@ def calculate_base_path!(my_options, manifest_path = nil)
           config_file = options[:config]
           if manifest_path.nil?
             if config_file.nil?
-              # If no config file is given, use the current working directory
-              # as the base path.
               Pathname.pwd
             else
-              # If a config file is given, use it as the config file
-              # and use its directory as the base path.
               Pathname.new(config_file).dirname
             end
           elsif File.file?(manifest_path) && config_file.nil?
-            # If a file is given, use it as the config file
-            # and use its directory as the base path.
             my_options["config"] = manifest_path
             Pathname.new(manifest_path).dirname
           else
-            # If directory is given, use it as the base path.
             Pathname.new(manifest_path)
           end
         end

lib/metanorma/cli/commands/template_repo.rb

@@ -1,6 +1,3 @@
-require "metanorma/cli/template_repo"
-require "metanorma/cli/thor_with_config"
-
 module Metanorma
   module Cli
     module Commands
@@ -11,7 +8,7 @@ class TemplateRepo < ThorWithConfig
                            desc: "Overwrite existing template"
 
         def add(name, source)
-          Metanorma::Cli::TemplateRepo.add(name, source, options)
+          Cli::TemplateRepo.add(name, source, options)
           UI.say("Template repo: #{name} has been added successfully")
         rescue Errors::DuplicateTemplateError
           UI.error("Duplicate metanorma template")