Refactor: autoload, OOP cleanup, and comprehensive spec improvements

Author: ronaldtse

.github/workflows/rubocop.yml

@@ -45,4 +45,4 @@ jobs:
           ruby-version: ${{ env.RUBY_VERSION }}
       - uses: reclaim-the-stack/rubocop-action@v1.1.0
         with:
-          gem_versions: rubocop:1.72.2 rubocop-performance:1.24.0
+          gem_versions: rubocop:1.86.0 rubocop-performance:1.26.1

.rspec

@@ -1,3 +1,3 @@
 --color
---profile
+--format progress
 --require spec_helper

CLAUDE.md

@@ -0,0 +1,97 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+metanorma-cli is the command-line interface for the Metanorma document processing toolchain. It provides the `metanorma` executable for compiling Metanorma documents across multiple standard flavors (ISO, IEC, IETF, BIPM, ITU, OGC, IEEE, JIS, etc.). It wraps the `metanorma` gem and adds CLI-level features: document compilation, collection rendering, site generation, template management, and STS XML conversion via `mnconvert`.
+
+## Build & Development Commands
+
+```bash
+# Install dependencies
+bundle install
+
+# Run full test suite
+bundle exec rake
+
+# Run a single spec file
+bundle exec rspec spec/metanorma/cli/compiler_spec.rb
+
+# Run a single example by line number
+bundle exec rspec spec/metanorma/cli/compiler_spec.rb:42
+
+# Lint
+bundle exec rubocop
+
+# Run the CLI locally
+bundle exec exe/metanorma compile document.adoc -t iso
+
+# IRB console with project loaded
+bin/console
+```
+
+The project uses direnv + nix flake (see `flake.nix`) for environment setup. If using nix, run `direnv allow` to enter the dev shell.
+
+Tests use RSpec with VCR cassettes (for HTTP stubbing), webmock, and rspec-command. SimpleCov is configured for coverage. Test state persistence is in `.rspec_status`.
+
+## Architecture
+
+### Entry Points
+
+- **`exe/metanorma`** — Main CLI executable. Sets up encoding, handles SOCKS proxy support via `socksify`, then delegates to `Metanorma::Cli.start(ARGV)`.
+- **`exe/metanorma-manifest`** — Standalone tool that converts YAML manifests into Metanorma XML/HTML collection indexes.
+- **`lib/metanorma-cli.rb`** — Just requires `metanorma/cli`.
+- **`lib/metanorma/cli.rb`** — Core module. `Cli.start` intercepts arguments: if no known command is found, it prepends `"compile"` (so bare `metanorma foo.adoc` works). Defines config paths (`~/.metanorma/config.yml` global, `.metanorma/config.yml` local).
+
+### Command System (Thor)
+
+- **`Command`** (`lib/metanorma/cli/command.rb`) — Main Thor command class. Defines all top-level commands: `compile`, `collection`, `convert`, `version`, `new`, `log_messages`, `list-extensions`, `list-doctypes`, `export-config`. Also has three subcommands: `template-repo`, `site`, `config`.
+- **`ThorWithConfig`** (`lib/metanorma/cli/thor_with_config.rb`) — Base class for commands. Overrides `options` to merge CLI arguments with values from global and local config files (`~/.metanorma/config.yml` and `.metanorma/config.yml`). The merging is done by `Commands::Config.load_configs`.
+- **`flavor.rb`** (module, not the one in `lib/metanorma/`) — Mixed into `Command`. Provides flavor discovery, extension listing, doctype tables, backend version lookups, and the `export-config` command.
+
+### Key Classes
+
+- **`Compiler`** (`compiler.rb`) — Wraps `Metanorma::Compile`. Validates input file existence, normalizes options (splits extract/extensions on commas), and calls the metanorma core compile pipeline.
+- **`Collection`** (`collection.rb`) — Renders Metanorma collections from YAML or XML files. Delegates to `Metanorma::Collection.parse` for the actual processing.
+- **`SiteGenerator`** (`site_generator.rb`) — Generates an HTML site from a collection of Metanorma documents. Reads a `metanorma.yml` manifest (via `SiteManifest` models), compiles individual `.adoc` files and collection files, builds a Relaton collection index, then converts to HTML using Liquid templates.
+- **`Generator`** (`generator.rb`) — Creates new Metanorma documents from templates. Merges base templates (`templates/base/`) with type-specific templates downloaded from Git repos.
+- **`GitTemplate`** (`git_template.rb`) — Manages Git-hosted document templates. Clones template repos to `~/.metanorma/templates/git/`.
+- **`TemplateRepo`** (`template_repo.rb`) — Manages template repo registrations in the global config file.
+
+### Subcommands
+
+- **`Commands::Config`** (`commands/config.rb`) — `metanorma config get/set/unset`. Manages hierarchical YAML config (global + local). The `load_configs` class method merges global then local config into CLI options.
+- **`Commands::Site`** (`commands/site.rb`) — `metanorma site generate`. Resolves manifest paths, stylesheet/template paths, and delegates to `SiteGenerator`.
+- **`Commands::TemplateRepo`** (`commands/template_repo.rb`) — `metanorma template-repo add`. Registers template repos.
+
+### Flavor System
+
+- **`Metanorma::Flavor`** (`lib/metanorma/flavor.rb`) — Activates and loads all supported metanorma flavor gems (`metanorma-iso`, `metanorma-iec`, etc.). Activation happens at require time (line 77). Private gems (nist, ribose) fail silently.
+- **`Metanorma::SiteManifest`** (`lib/metanorma/site_manifest.rb`) — Lutaml::Model serializable classes for parsing `metanorma.yml` site manifest files.
+
+### Supporting Code
+
+- **`UI`** (`ui.rb`) — Thin Thor-based wrapper for console output (say, info, error, table).
+- **`Errors`** (`errors.rb`) — Custom error classes: `FileNotFoundError`, `FatalCompilationError`, `InvalidManifestFileError`, `DuplicateTemplateError`.
+- **`stringify_all_keys.rb`** — Core extensions for deep key stringification/symbolization on Hash and Array. Note: `metanorma-utils` provides similar functionality via `Metanorma::Utils::Hash`.
+
+## Configuration
+
+Config files are YAML with a `cli` key at the top level. Values from config are merged into Thor options (global first, then local). Example:
+
+```yaml
+cli:
+  agree_to_terms: true
+  install_fonts: false
+```
+
+- Global: `~/.metanorma/config.yml`
+- Local: `.metanorma/config.yml`
+
+## Testing Notes
+
+- Spec helper mocks `Mn2pdf.convert` and `MnConvert.convert` to copy files instead of running real conversions.
+- `strip_guid` helper normalizes UUIDs in XML output for deterministic comparisons.
+- Acceptance specs (`spec/acceptance/`) test the CLI end-to-end via `rspec-command`.
+- VCR cassettes are in `spec/vcr_cassettes/` for HTTP request stubs.

lib/metanorma/cli.rb

@@ -2,31 +2,32 @@
 
 require "metanorma"
 require "metanorma/flavor"
-
 require "metanorma/site_manifest"
 
-require "metanorma/cli/version"
-require "metanorma/cli/errors"
-require "metanorma/cli/command"
-
 module Metanorma
   module Cli
+    autoload :Collection, "metanorma/cli/collection"
+    autoload :Command, "metanorma/cli/command"
+    autoload :Commands, "metanorma/cli/commands"
+    autoload :Compiler, "metanorma/cli/compiler"
+    autoload :ConfigExporter, "metanorma/cli/config_exporter"
+    autoload :Errors, "metanorma/cli/errors"
+    autoload :FlavorMethods, "metanorma/cli/flavor"
+    autoload :Generator, "metanorma/cli/generator"
+    autoload :GitTemplate, "metanorma/cli/git_template"
+    autoload :SiteGenerator, "metanorma/cli/site_generator"
+    autoload :TemplateRepo, "metanorma/cli/template_repo"
+    autoload :ThorWithConfig, "metanorma/cli/thor_with_config"
+    autoload :UI, "metanorma/cli/ui"
+    autoload :VERSION, "metanorma/cli/version"
+
     CONFIG_DIRNAME = ".metanorma"
     CONFIG_FILENAME = "config.yml"
 
     def self.load_flavors
       Metanorma::Flavor.load_flavors
     end
 
-    # Invoking commands
-    #
-    # In the Metanorma CLI, we've included some custom behavior,
-    # like exposing the compiation directly from the root command.
-    #
-    # So, for this use case we first check if the user is actually
-    # trying to compile a document or not, and based on that we'll
-    # compile the document or show the help documentation.
-    #
     def self.start(arguments)
       if find_command(arguments).empty?
         arguments.unshift("compile")

lib/metanorma/cli/collection.rb

@@ -16,6 +16,11 @@ def self.render(filename, options = {})
         new(filename, options).render
       end
 
+      def self.parse_formats(formats)
+        formats = formats.split(",") if formats.is_a?(String)
+        (formats || []).map { |extension| extension.strip.to_sym }
+      end
+
       def render
         extract_options_from_file
         collection_file.render(collection_options.compact)
@@ -37,7 +42,7 @@ def collection_options
           compile: @compile_options,
           output_folder: build_output_folder,
           coverpage: options.fetch(:coverpage, nil),
-          format: collection_output_formats(options.fetch(:format, "")),
+          format: self.class.parse_formats(options.fetch(:format, "")),
           site_generate: options["site_generate"],
         }
       end
@@ -54,17 +59,9 @@ def build_output_folder
         end
       end
 
-      def collection_output_formats(formats)
-        if formats.is_a?(String)
-          formats = formats.split(",")
-        end
-
-        (formats || []).map { |extension| extension.strip.to_sym }
-      end
-
       def extract_options_from_file
         yaml_file = if /\.ya?ml$/.match?(file.to_s)
-                      YAML.safe_load(File.read(file.to_s))
+                      YAML.safe_load_file(file.to_s)
                     elsif /\.xml$/.match?(file.to_s)
                       xml_extract_options_from_file
                     end

lib/metanorma/cli/command.rb

@@ -1,22 +1,15 @@
 # frozen_string_literal: true
 
-require "metanorma/cli/compiler"
-require "metanorma/cli/generator"
-require "metanorma/cli/collection"
-require "metanorma/cli/git_template"
-require "metanorma/cli/thor_with_config"
-require "metanorma/cli/commands/config"
-require "metanorma/cli/commands/template_repo"
-require "metanorma/cli/commands/site"
 require "mnconvert"
-require_relative "flavor"
 
 module Metanorma
   module Cli
     class Command < ThorWithConfig
+      include FlavorMethods
+
       class_option :progress, aliases: "-s", type: :boolean, default: false,
-                              desc: "Show progress for long running tasks" \
-                                    " (like download)"
+                              desc: "Show progress for long running tasks " \
+                                    "(like download)"
 
       desc "new NAME", "Create new Metanorma document"
       option :type, aliases: "-t", required: true, desc: "Document type"
@@ -40,28 +33,28 @@ def new(name)
       option :wrapper, aliases: "-w", type: :boolean,
                        desc: "Create wrapper folder for HTML output"
       option :asciimath, aliases: "-a", type: :boolean,
-                         desc: "Keep Asciimath in XML output instead of" \
-                               " converting it to MathM"
+                         desc: "Keep Asciimath in XML output instead of " \
+                               "converting it to MathM"
       option :datauriimage, aliases: "-d", type: :boolean,
                             desc: "Encode HTML output images as data URIs"
       option :relaton, aliases: "-R",
-                       desc: "Export Relaton XML for document to nominated" \
-                             " filename"
+                       desc: "Export Relaton XML for document to nominated " \
+                             "filename"
       option :extract, aliases: "-e",
-                       desc: "Export sourcecode fragments from this document" \
-                             " to nominated directory"
+                       desc: "Export sourcecode fragments from this document " \
+                             "to nominated directory"
       option :version, aliases: "-v",
                        desc: "Print version of code (accompanied with -t)"
       option :log_messages, aliases: "-L",
                             desc: "Display available log messages " \
-                            "(accompanied with -t)"
+                                  "(accompanied with -t)"
       option :output_dir, aliases: "-o",
                           desc: "Directory to save compiled files"
       option :strict, aliases: "-S", type: :boolean,
                       desc: "Strict compilation: abort if there are any errors"
       option :agree_to_terms,
              type: :boolean,
-             desc: "Agree / Disagree with all third-party licensing terms "\
+             desc: "Agree / Disagree with all third-party licensing terms " \
                    "presented (WARNING: do know what you are agreeing with!)"
       option :install_fonts, type: :boolean, default: true,
                              desc: "Install required fonts"
@@ -99,14 +92,14 @@ def compile(file_name = nil)
       option :coverpage, aliases: "-c", desc: "Liquid template"
       option :agree_to_terms,
              type: :boolean,
-             desc: "Agree / Disagree with all third-party licensing terms "\
+             desc: "Agree / Disagree with all third-party licensing terms " \
                    "presented (WARNING: do know what you are agreeing with!)"
       option :install_fonts, type: :boolean, default: true,
                              desc: "Install required fonts"
       option :continue_without_fonts,
              type: :boolean,
              desc: "Continue processing even when fonts are missing"
-      option :strict, aliases: "-S", type: :boolean, \
+      option :strict, aliases: "-S", type: :boolean,
                       desc: "Strict compilation: abort if there are any errors"
 
       def collection(filename = nil)
@@ -193,7 +186,7 @@ def list_doctypes(type = nil)
 
       desc "export-config", "Export flvor configuration"
       def export_config(type = nil)
-        export_config_flavor(type)
+        ConfigExporter.new(type).export
       end
 
       desc "template-repo", "Manage metanorma templates repository"

lib/metanorma/cli/commands/config.rb

@@ -1,8 +1,6 @@
 require "pathname"
 require "metanorma-utils"
 
-# require "metanorma/cli/stringify_all_keys"
-
 module Metanorma
   module Cli
     module Commands
@@ -48,17 +46,6 @@ def unset(name)
 
         def self.exit_on_failure?() true end
 
-        # priority:
-        # IDEAL:
-        # thor defaults -> global conf -> local conf
-        #   -> env vars -> passed arguments
-        #
-        # ACTUAL:
-        # all arguments -> global conf -> local conf
-        #
-        # - thor doesn't provide to differentiate default values against passed
-        #   args
-        # - thor doesn't allow to get all args available for current command
         def self.load_configs(options, configs = [
           Metanorma::Cli.global_config_path, Metanorma::Cli.local_config_path
         ])
@@ -108,19 +95,15 @@ def dig_path(str)
         end
 
         def deep_set(hash, value, *keys)
-          keys[0...-1].reduce(hash) do |acc, h|
-            tmp = acc.public_send(:[], h)
-            if tmp.nil?
-              acc[h] = tmp = Hash.new
-            end
-            tmp
-          end.public_send(:[]=, keys.last, value)
+          leaf = keys[0...-1].reduce(hash) do |acc, k|
+            acc[k] ||= {}
+          end
+          leaf[keys.last] = value
         end
 
         def deep_unset(hash, *keys)
-          keys[0...-1].reduce(hash) do |acc, h|
-            acc.public_send(:[], h)
-          end.delete(keys.last)
+          leaf = keys[0...-1].reduce(hash) { |acc, k| acc[k] }
+          leaf.delete(keys.last)
         end
 
         def try_convert_to_bool(value)