[FEATURE] basic schema and data validation [cuelang]

[noelmcloughlin]

Overview

There is no Data Constraints languages used in Saltstack formulas. If these were used, as part of CI/CD, then it would help formula developers, and provide example of how users could add/introduce data validation themselves. A related issue, is that no saltstack formula has a published schema, except what is available in README, pillar.example, etc.

To start a discussion I would suggest using Cuelang (https://cuelang.org/docs/install) to address three concerns:

• introduce Schema for formula (without forcing it)
• Check formula against Schema during CI/CD (without forcing it) # needs gitlab-ci updates
• Improve quality assurance as a feature

DEMO

1. install cuelang from https://cuelang.org/docs/install/

2. goto a directory with YAML files

        $ git clone https://github.com/saltstack-formulas/template-formula
        $ cd template-formula/TEMPLATE/parameters/
   

3. create a Schema file named schema.cue in this directory.

       #template: {
           pkg?: name?: string
           rootgroup?: string
           hide_output?: bool
           dir_mode?: =~"^0?[124567]{3}$" // any mode of length 3, with 0 prefix optional
           mode?: =~"^0?[124567]{3}$" // any mode of length 3, with 0 prefix optional
           config?:    string
           service?: name?: string
           subcomponent?: config: string
       
           // legacy
           pip_pkg?: string
           pkgs_add?: [...]
           pips?: [...]
       
           // Just here for testing
           added_in_defaults?: string
           winner?:            string
               ...
       }
       values?: {...#template}     // probable yaml namespace
   

4. Make a change to some YAML file to introduce a bad value.

5. Validate against schema data-constraints (cue vet FILE.yaml schema.cue)

            $ for f in $( find . | grep yaml$); do cue vet $f schema.cue; done
            values.pkg.name: conflicting values 111 and string (mismatched types int and string):
                .\os\Fedora.yaml:16:12
                .\schema.cue:10:18
                .\schema.cue:29:14
   

6. Fix the error in your YAML and try again.

            $ for f in $( find . | grep yaml$); do cue vet $f schema.cue; done
            $ echo $?
             0
   

Related issues

Salt project: saltstack/salt#54193

[noelmcloughlin]

@myii regarding Dhall. There is overlap with cue: cuelang/cue#669 (comment)

My PR is to introduce basic Schema support without forcing cue. The help I need from you is to update your CI/CD image to install Cuelang (and Dhall perhaps). A basic pipeline job can check yaml against schema.cue schema (its only 1% coverage because I just wanted to showcase the possibility). The code footprint is tiny - one file.

A Dhall footprint could be introduced in different PR but neither dhall or cue should be forced.