TCLI-107 add proper subcommand support

Signed-off-by: Sean Corfield <sean@corfield.org>

Author: seancorfield

CHANGELOG.md

@@ -1,5 +1,8 @@
 # Change Log
 
+* Release 1.4.next in progress
+  * Enhance support for subcommand option parsing: add `:subcommand :explicit` and `:subcommand :implicit` options. The former replaces `:in-order true` (which is deprecated) and the latter expands that parsing to treat an unknown option as starting a new subcommand.
+
 * Release 1.3.250 2025-12-30
   * Update parent pom and Clojure dependency version
 

doc/new-in-0-4.md

@@ -33,6 +33,8 @@ options. To aid in designing such programs, `clojure.tools.cli/parse-opts`
 accepts an `:in-order` option that directs it to stop processing arguments at
 the first unrecognized token.
 
+_As of 1.4.next, the `:in-order` option is deprecated and replaced by the `:subcommand` option with values `:explicit` and `:implicit`._
+
 For instance, the `git` program has a set of top-level options that are
 unrecognized by subcommands and vice-versa:
 

doc/parse-opts.md

@@ -188,10 +188,6 @@ this into complete documentation for the library, with examples, over time):
   A few function options may be specified to influence the behavior of
   parse-opts:
 
-    :in-order     Stop option processing at the first unknown argument. Useful
-                  for building programs with subcommands that have their own
-                  option specs.
-
     :no-defaults  Only include option values specified in arguments and do not
                   include any default values in the resulting options map.
                   Useful for parsing options from multiple sources; i.e. from a
@@ -201,6 +197,12 @@ this into complete documentation for the library, with examples, over time):
                   matches any other option, it is considered to be missing (and
                   you have a parse error).
 
+    :subcommand   Stop option processing at the first unknown argument. Useful
+                  for building programs with subcommands that have their own
+                  option specs. Can be set to :explicit or :implicit. :explicit
+                  requires a non-option (explicit) subcommand argument to
+                  trigger collection of subcommand arguments. :implicit treats an unknown option as starting a new subcommand.
+
     :summary-fn   A function that receives the sequence of compiled option specs
                   (documented at #'clojure.tools.cli/compile-option-specs), and
                   returns a custom option summary string.

src/main/clojure/clojure/tools/cli.cljc

@@ -35,10 +35,10 @@
   Long options with `=` are always parsed as option + optarg, even if nothing
   follows the `=` sign.
 
-  If the :in-order flag is true, the first non-option, non-optarg argument
+  If the :subcommand option is present, the first non-option, non-optarg argument
   stops options processing. This is useful for handling subcommand options."
   [required-set args & options]
-  (let [{:keys [in-order]} (apply hash-map options)]
+  (let [{:keys [subcommand]} (apply hash-map options)]
     (loop [opts [] argv [] [car & cdr] args]
       (if car
         (condp re-seq car
@@ -66,7 +66,7 @@
                                         (recur (conj os [:short-opt o]) cs)
                                         [(conj os [:short-opt o]) cdr]))))]
                    (recur (into opts os) argv cdr))
-          (if in-order
+          (if subcommand
             (recur opts (into argv (cons car cdr)) [])
             (recur opts (conj argv car) cdr)))
         [opts argv]))))
@@ -282,59 +282,67 @@
   If the :no-defaults flag is true, only options specified in the tokens are
   included in the option-map.
 
-  Unknown options, missing options, missing required arguments, option
-  argument parsing exceptions, and validation failures are collected into
-  a vector of error message strings.
+  By default, unknown options, missing options, missing required arguments,
+  option argument parsing exceptions, and validation failures are collected
+  into a vector of error message strings.
+
+  If :subcommand :implicit is provided, unknown options trigger in-order
+  processing, collecting the rest of the tokens as positional arguments.
 
   If the :strict flag is true, required arguments that match other options
   are treated as missing, instead of a literal value beginning with - or --.
 
   Returns [option-map error-messages-vector]."
   [specs tokens & options]
-  (let [{:keys [no-defaults strict]} (apply hash-map options)
+  (let [{:keys [no-defaults strict subcommand]} (apply hash-map options)
         defaults (default-option-map specs :default)
         default-fns (default-option-map specs :default-fn)
-        requireds (missing-errors specs)]
+        requireds (missing-errors specs)
+        collecting (atom false)]
     (-> (reduce
-          (fn [[m ids errors] [opt-type opt optarg]]
-            (if-let [spec (find-spec specs opt-type opt)]
-              (let [[value error] (parse-optarg spec opt optarg)
-                    id (:id spec)]
-                (if-not (= value ::error)
-                  (if (and strict
-                           (or (find-spec specs :short-opt optarg)
-                               (find-spec specs :long-opt optarg)))
-                    [m ids (conj errors (missing-required-error opt (:required spec)))]
-                    (let [m' (if-let [update-fn (:update-fn spec)]
-                               (if (:multi spec)
-                                 (update m id update-fn value)
-                                 (update m id update-fn))
-                               ((:assoc-fn spec assoc) m id value))]
-                      (if (:post-validation spec)
-                        (let [[value error] (validate (get m' id) spec opt optarg)]
-                          (if (= value ::error)
-                            [m ids (conj errors error)]
-                            [m' (conj ids id) errors]))
-                        [m' (conj ids id) errors])))
-                  [m ids (conj errors error)]))
-              [m ids (conj errors (str "Unknown option: " (pr-str opt)))]))
-          [defaults [] []] tokens)
+         (fn [[m ids errors args] [opt-type opt optarg]]
+           (if-let [spec (when-not @collecting (find-spec specs opt-type opt))]
+             (let [[value error] (parse-optarg spec opt optarg)
+                   id (:id spec)]
+               (if-not (= value ::error)
+                 (if (and strict
+                          (or (find-spec specs :short-opt optarg)
+                              (find-spec specs :long-opt optarg)))
+                   [m ids (conj errors (missing-required-error opt (:required spec))) args]
+                   (let [m' (if-let [update-fn (:update-fn spec)]
+                              (if (:multi spec)
+                                (update m id update-fn value)
+                                (update m id update-fn))
+                              ((:assoc-fn spec assoc) m id value))]
+                     (if (:post-validation spec)
+                       (let [[value error] (validate (get m' id) spec opt optarg)]
+                         (if (= value ::error)
+                           [m ids (conj errors error)]
+                           [m' (conj ids id) errors]))
+                       [m' (conj ids id) errors args])))
+                 [m ids (conj errors error) args]))
+             (if (or @collecting (= :implicit subcommand))
+               (do
+                 (reset! collecting true)
+                 [m ids errors (conj args opt)])
+               [m ids (conj errors (str "Unknown option: " (pr-str opt))) args])))
+         [defaults [] [] []] tokens)
         (#(reduce
-           (fn [[m ids errors] [id error]]
+           (fn [[m ids errors args] [id error]]
              (if (contains? m id)
-               [m ids errors]
-               [m ids (conj errors error)]))
+               [m ids errors args]
+               [m ids (conj errors error) args]))
            % requireds))
         (#(reduce
-           (fn [[m ids errors] [id f]]
+           (fn [[m ids errors args] [id f]]
              (if (contains? (set ids) id)
-               [m ids errors]
-               [(assoc m id (f (first %))) ids errors]))
+               [m ids errors args]
+               [(assoc m id (f (first %))) ids errors args]))
            % default-fns))
-        (#(let [[m ids errors] %]
+        (#(let [[m ids errors args] %]
             (if no-defaults
-              [(select-keys m ids) errors]
-              [m errors]))))))
+              [(select-keys m ids) errors args]
+              [m errors args]))))))
 
 (defn make-summary-part
   "Given a single compiled option spec, turn it into a formatted string,
@@ -588,10 +596,6 @@
   A few function options may be specified to influence the behavior of
   parse-opts:
 
-    :in-order     Stop option processing at the first unknown argument. Useful
-                  for building programs with subcommands that have their own
-                  option specs.
-
     :no-defaults  Only include option values specified in arguments and do not
                   include any default values in the resulting options map.
                   Useful for parsing options from multiple sources; i.e. from a
@@ -601,19 +605,39 @@
                   matches any other option, it is considered to be missing (and
                   you have a parse error).
 
+    :subcommand   Stop option processing at the first unknown argument. Useful
+                  for building programs with subcommands that have their own
+                  option specs. Can be set to :explicit or :implicit. :explicit
+                  requires a non-option (explicit) subcommand argument to
+                  trigger collection of subcommand arguments. :implicit treats an
+                  unknown option as starting a new subcommand.
+
     :summary-fn   A function that receives the sequence of compiled option specs
                   (documented at #'clojure.tools.cli/compile-option-specs), and
                   returns a custom option summary string.
   "
   [args option-specs & options]
-  (let [{:keys [in-order no-defaults strict summary-fn]} (apply hash-map options)
+  (let [{:keys [in-order no-defaults strict subcommand summary-fn]}
+        (apply hash-map options)
+        ;; handle deprecation of in-order here:
+        subcommand (if subcommand
+                     (do
+                       (when in-order
+                         (throw (ex-info ":in-order is deprecated and cannot be used if :subcommand is present"
+                                         {:in-order   in-order
+                                          :subcommand subcommand})))
+                       subcommand)
+                     (when in-order :explicit))
         specs (compile-option-specs option-specs)
         req (required-arguments specs)
-        [tokens rest-args] (tokenize-args req args :in-order in-order)
-        [opts errors] (parse-option-tokens specs tokens
-                                           :no-defaults no-defaults :strict strict)]
+        [tokens rest-args] (tokenize-args req args :subcommand subcommand)
+        [opts errors implicit-args]
+        (parse-option-tokens specs tokens
+                             :no-defaults no-defaults
+                             :strict      strict
+                             :subcommand  subcommand)]
     {:options opts
-     :arguments rest-args
+     :arguments (into rest-args implicit-args)
      :summary ((or summary-fn summarize) specs)
      :errors (when (seq errors) errors)}))
 

src/test/clojure/clojure/tools/cli_test.cljc

@@ -6,7 +6,13 @@
 ;; Refer private vars
 (def tokenize-args        #'cli/tokenize-args)
 (def compile-option-specs #'cli/compile-option-specs)
-(def parse-option-tokens  #'cli/parse-option-tokens)
+(def parse-option-tokens' #'cli/parse-option-tokens)
+(defn- parse-option-tokens
+  "To avoid changing all the tests that assume parse-option-tokens returns
+   only [opts errors]"
+  [specs tokens & options]
+  (let [[opts errors _args] (apply #'cli/parse-option-tokens specs tokens options)]
+    [opts errors]))
 
 (deftest test-tokenize-args
   (testing "expands clumped short options"
@@ -20,10 +26,12 @@
   (testing "stops option processing on double dash"
     (is (= (tokenize-args #{} ["-a" "--" "-b"])
            [[[:short-opt "-a"]] ["-b"]])))
-  (testing "finds trailing options unless :in-order is true"
+  (testing "finds trailing options unless :subcommand is present"
     (is (= (tokenize-args #{} ["-a" "foo" "-b"])
            [[[:short-opt "-a"] [:short-opt "-b"]] ["foo"]]))
-    (is (= (tokenize-args #{} ["-a" "foo" "-b"] :in-order true)
+    (is (= (tokenize-args #{} ["-a" "foo" "-b"] :subcommand :explicit)
+           [[[:short-opt "-a"]] ["foo" "-b"]]))
+    (is (= (tokenize-args #{} ["-a" "foo" "-b"] :subcommand :implicit)
            [[[:short-opt "-a"]] ["foo" "-b"]])))
   (testing "does not interpret single dash as an option"
     (is (= (tokenize-args #{} ["-"]) [[] ["-"]]))))
@@ -100,7 +108,7 @@
                      #?(:clj (binding [*err* *out*]
                                (compile-option-specs [[nil "--alpha" :validate nil :flag true]]))
                         :cljr (binding [*err* *out*]
-                                (compile-option-specs [[nil "--alpha" :validate nil :flag true]]))							   
+                                (compile-option-specs [[nil "--alpha" :validate nil :flag true]]))
                         :cljs (binding [*print-err-fn* *print-fn*]
                                 (compile-option-specs [[nil "--alpha" :validate nil :flag true]]))))))
       (is (re-find #"Warning:.* :validate"
@@ -422,11 +430,47 @@
            ["foo" "bar" "-b" "baz"])))
   (testing "provides an option summary at :summary"
     (is (re-seq #"-a\W+--alpha" (:summary (parse-opts [] [["-a" "--alpha"]])))))
+  ;; deprecated:
   (testing "processes arguments in order when :in-order is true"
     (is (= (:arguments (parse-opts ["-a" "foo" "-b"]
                                    [["-a" "--alpha"] ["-b" "--beta"]]
                                    :in-order true))
            ["foo" "-b"])))
+  (testing "processes arguments in order when :subcommand is present"
+    (is (= (:arguments (parse-opts ["-a" "foo" "-b"]
+                                   [["-a" "--alpha"] ["-b" "--beta"]]
+                                   :subcommand :explicit))
+           ["foo" "-b"]))
+    (is (= (:arguments (parse-opts ["-a" "foo" "-b"]
+                                   [["-a" "--alpha"] ["-b" "--beta"]]
+                                   :subcommand :implicit))
+           ["foo" "-b"]))
+    ;; explicit subcommand requires at least one non-option argument:
+    (is (= (:arguments (parse-opts ["-a" "-b"]
+                                   [["-a" "--alpha"]]
+                                   :subcommand :explicit))
+           []))
+    (is (= (:errors (parse-opts ["-a" "-b"]
+                                [["-a" "--alpha"]]
+                                :subcommand :explicit))
+           ["Unknown option: \"-b\""]))
+    ;; implicit subcommand is triggered by an unknown option:
+    (is (= (:arguments (parse-opts ["-a" "-b"]
+                                   [["-a" "--alpha"]]
+                                   :subcommand :implicit))
+           ["-b"]))
+    (is (= (:errors (parse-opts ["-a" "-b"]
+                                [["-a" "--alpha"]]
+                                :subcommand :implicit))
+           nil))
+    (is (= (:arguments (parse-opts ["-a" "-b"]
+                                   [["-b" "--beta"]]
+                                   :subcommand :implicit))
+           ["-a" "-b"]))
+    (is (= (:errors (parse-opts ["-a" "-b"]
+                                [["-b" "--beta"]]
+                                :subcommand :implicit))
+           nil)))
   (testing "does not merge over default values when :no-defaults is true"
     (let [option-specs [["-p" "--port PORT" :default 80]
                         ["-H" "--host HOST" :default "example.com"]