Deployment: Multiple Files, Hot Reload, Dynamic for K8S

[predic8]

We want:

to split a configuration in multiple files:

Requirements:

• Easy to explain
• No surprises

Problems:

• Order of proxy rules

Ideas:

• Import from apis.yaml, sequence is determined by sequence of imports
  • Repeatable strict order
  • Subdirs are no question
• Load all *.apis.yaml from conf or depending on startup the local folder
  • apis.yaml is last, within a yaml, the order is top to bottom.
  • To add an api, its just add a file.
• Prios:
  • 1 Highest, Default 100, unlimited Lowest.
  • Compatible with K8S
• 100-fasf.apis.yaml, 200-dfsa.apis.yaml, 300-afsa.apis.yaml
  Sorting

[christiangoerdes]

We could start with adding a new top-level include list. The include order defines the rule order.

include:
   - demo.apis.yaml  # Prio 1
   - demo2.apis.yaml # Prio 2
   - ./openapi       # Loads all *.apis.yaml from this direcotry

There are several things to decide:

• When including a directory:
  • In what order are the *.apis.yaml loaded? (TB: No specified order)
  • Only include all *.apis.yaml or also all *.yml (TB: *.apis.y(a)ml)
• Do the rules from the "main" file (the file that includes others) come before or after the included ones? (TB main is last => lowest prio)
• Can you reference components from files that are not included (for example, to keep all components in a separate file)? (TB not now)
• Is it allowed to define components in included files? (TB yes)
  • If yes, how are they referenced from the main file ($ref: "#/components/..." vs. $ref: "other.yaml#/components/...")? (TB id must be unique over entire configuration)
• Are all files equally powerful (both included and including files)?
  • If yes, all files can include other files and define component (see the problem above). In that case, only the entry file matters (The file that is started by the membrane.sh).
    • Circular includes would be possible --> ERROR
  • If not, there is one main file and the included files which cannot include other files or define components.
    • This would require a second schema
    • We would then have two different config types to maintain, document, and explain elsewhere.
• What happens when you include the same file multiple times? (TB Loaded only once)
• What happens when multiple files define a global? (TB 2 global in one file? Only in main)
• What happens when multiple files define a configuration (TB Only main, warn that others are ignored)
• Hot Reload: Restart/Reload all files? (TB later yes)
• What is the basePath when an included file (conf/apis/x.apis.yaml) references groovy scripts or XSLT stylesheets?
  • The one from main?
  • The location of the included file (basePath is requested from interceptors by calling router.getBasePath())

Other TODOS:

• Handle error-output for errors from included files
• For debugging: method/log/print to show the effective configuration created from the config and includes. (TB No, just the name of the API)

[predic8]

@rrayst comment plz