|
| 1 | +## Goal |
| 2 | + |
| 3 | +Design a robust, deterministic strategy to inline all JSON Schema $ref occurrences into their target schemas for docs JSON produced by goa. The generator only uses local references to the top-level definitions object (paths of the form "#/definitions/<Name>"). |
| 4 | + |
| 5 | +## Where $ref appear in goa’s output |
| 6 | + |
| 7 | +- **User/Result types**: Type schemas reference definitions via `TypeRef*` and `ResultTypeRef*`. |
| 8 | +- **Object properties**: Nested fields can be `$ref`. |
| 9 | +- **Array items**: `items` may be a `$ref` when the element type is a user/result type. |
| 10 | +- **Map values**: `additionalProperties` may be a schema containing `$ref` (or boolean `true`). |
| 11 | +- **Union types**: `anyOf` is an array of schemas that may contain `$ref`. |
| 12 | +- **Service-level top schema**: APISchema properties refer to `#/definitions/<Service>`. |
| 13 | + |
| 14 | +Assumption: Only references to `#/definitions/<Name>` are present. No external files/URLs. |
| 15 | + |
| 16 | +## Constraints |
| 17 | + |
| 18 | +- Definitions are in a per-root map; keys are type names. |
| 19 | +- Definitions can themselves contain nested `$ref`. |
| 20 | +- Must avoid infinite recursion on cyclical definitions. |
| 21 | +- Preserve required fields, examples, and order where applicable. |
| 22 | + |
| 23 | +## Strategy (high-level) |
| 24 | + |
| 25 | +1) **Snapshot definitions** |
| 26 | + - Work on a deep-copied map of the per-root `definitions` to avoid mutating shared state. |
| 27 | + |
| 28 | +2) **Inlining pass** |
| 29 | + - Implement `inlineRefs(schema, defs, stack)`: |
| 30 | + - If `schema.Ref == "#/definitions/<Name>"`: |
| 31 | + - If `<Name>` is in `stack`, keep `$ref` (cycle break) and return. |
| 32 | + - Else: push `<Name>`, deep-copy `defs[Name]`, recursively `inlineRefs` on the copy, then replace current node with the copy and clear `Ref`. Pop `<Name>`. |
| 33 | + - If `schema.Ref == ""`: recursively process all composite positions: |
| 34 | + - `properties` values |
| 35 | + - `items` |
| 36 | + - `additionalProperties` when it is a `*Schema` |
| 37 | + - `anyOf` elements |
| 38 | + - (If present) nested `definitions` for completeness |
| 39 | + |
| 40 | +3) **Apply order** |
| 41 | + - Perform schema transforms that affect keys first (e.g., JSON tag rename and required filtering), then inline. |
| 42 | + |
| 43 | +4) **Service-level application** |
| 44 | + - Run `inlineRefs` on every reachable schema under services (payload, result, streaming payload/result, error types). |
| 45 | + - Optionally inline inside `definitions` if you plan to drop them entirely. |
| 46 | + |
| 47 | +5) **Cycles** |
| 48 | + - Use a `stack` (set of definition names being expanded). If a name is already in the stack, retain `$ref` at that edge to prevent infinite expansion. This yields minimal `$ref` for strongly-cyclic graphs. |
| 49 | + |
| 50 | +6) **AnyOf** |
| 51 | + - Goa builds `anyOf` by appending union variants in-order. Inline each element; preserve order. |
| 52 | + |
| 53 | +7) **Maps** |
| 54 | + - If `additionalProperties` is `true`, leave as is. If it is a schema, inline there as well. |
| 55 | + |
| 56 | +8) **Performance** |
| 57 | + - Start without memoization. If profiling reveals hotspots, consider caching fully inlined, deep-copied definitions by name and reusing them, taking care not to share mutable pointers unexpectedly. |
| 58 | + |
| 59 | +9) **Post-processing** |
| 60 | + - If you require a completely `$ref`-free document, remove `definitions` after inlining all reachable schemas. Otherwise, keep or prune unused definitions based on tests/goldens. |
| 61 | + |
| 62 | +## Pseudocode |
| 63 | + |
| 64 | +```go |
| 65 | +func inlineAllServiceSchemas(d *data, defs map[string]*openapi.Schema) { |
| 66 | + stack := map[string]bool{} |
| 67 | + for _, s := range d.Services { |
| 68 | + for _, m := range s.Methods { |
| 69 | + if m.Payload != nil && m.Payload.Type != nil { |
| 70 | + inlineRefs(m.Payload.Type, defs, stack) |
| 71 | + } |
| 72 | + if m.StreamingPayload != nil && m.StreamingPayload.Type != nil { |
| 73 | + inlineRefs(m.StreamingPayload.Type, defs, stack) |
| 74 | + } |
| 75 | + if m.Result != nil && m.Result.Type != nil { |
| 76 | + inlineRefs(m.Result.Type, defs, stack) |
| 77 | + } |
| 78 | + if m.StreamingResult != nil && m.StreamingResult.Type != nil { |
| 79 | + inlineRefs(m.StreamingResult.Type, defs, stack) |
| 80 | + } |
| 81 | + for _, e := range m.Errors { |
| 82 | + if e.Type != nil { |
| 83 | + inlineRefs(e.Type, defs, stack) |
| 84 | + } |
| 85 | + } |
| 86 | + } |
| 87 | + } |
| 88 | +} |
| 89 | + |
| 90 | +func inlineRefs(s *openapi.Schema, defs map[string]*openapi.Schema, stack map[string]bool) { |
| 91 | + if s == nil { |
| 92 | + return |
| 93 | + } |
| 94 | + if s.Ref != "" { |
| 95 | + const prefix = "#/definitions/" |
| 96 | + if !strings.HasPrefix(s.Ref, prefix) { return } |
| 97 | + name := strings.TrimPrefix(s.Ref, prefix) |
| 98 | + if stack[name] { return } |
| 99 | + def, ok := defs[name] |
| 100 | + if !ok || def == nil { return } |
| 101 | + stack[name] = true |
| 102 | + copy := dupSchema(def) |
| 103 | + inlineRefs(copy, defs, stack) |
| 104 | + *s = *copy |
| 105 | + s.Ref = "" |
| 106 | + delete(stack, name) |
| 107 | + return |
| 108 | + } |
| 109 | + for _, p := range s.Properties { inlineRefs(p, defs, stack) } |
| 110 | + if s.Items != nil { inlineRefs(s.Items, defs, stack) } |
| 111 | + if ap, ok := s.AdditionalProperties.(*openapi.Schema); ok && ap != nil { |
| 112 | + inlineRefs(ap, defs, stack) |
| 113 | + } |
| 114 | + for _, a := range s.AnyOf { inlineRefs(a, defs, stack) } |
| 115 | + for _, d := range s.Definitions { inlineRefs(d, defs, stack) } |
| 116 | +} |
| 117 | +``` |
| 118 | + |
| 119 | +## Integration points in the plugin |
| 120 | + |
| 121 | +- After building `docs` and `defs`, and after JSON tag transforms: |
| 122 | + - Gate behind `InlineRefs` option: call `inlineAllServiceSchemas(docs, defs)`. |
| 123 | + - Decide whether to keep or drop `docs.Definitions` based on goldens. If keeping, you may optionally prune unused definitions. |
| 124 | + |
| 125 | +## Edge cases and guarantees |
| 126 | + |
| 127 | +- **Cycles**: Minimal `$ref` retained on cycle entry. |
| 128 | +- **Required and examples**: Preserved; JSON-tag transform already remaps them pre-inlining. |
| 129 | +- **Maps**: Free-form (`true`) untouched; schema-valued inlined. |
| 130 | +- **Unions**: Order preserved in `anyOf`. |
| 131 | + |
| 132 | +## Rationale |
| 133 | + |
| 134 | +- Tailored to the shapes produced by goa: only `#/definitions/` refs. |
| 135 | +- Deterministic and safe (deep copies, cycle guard), compositional (handles all container fields). |
| 136 | +- Clean pipeline: rename/tag first, inline second. |
| 137 | + |
| 138 | + |
0 commit comments