Improved documentation of complex query parameters (arrays, objects)

[romalytvynenko]

This PR improves documentation of complex query parameters (arrays and objects) so it better suits available OpenAPI tools. Currently OpenAPI specification (3.1) and available tools don't allow to easily document complex query parameters (see OAI/OpenAPI-Specification#1706).

Particularly, this PR adds handling of 2 cases.

1. Objects and nested objects (fixes #650)

Imagine you have the following validation rule for your query parameters:

$request->validate([
    'filter.name' => ['string'],
]);

You can then pass some value with the following query string:

?filter[name]=something

Previously Scramble documented this as filter parameter that is an object with the name property:

{
  "name": "filter", 
  "in": "query", 
  "schema": {
    "type": "object", 
    "properties": {
      "name": {"type": "string"}
    }
  }
}

This seems okay, but it didn't work properly in Try it out section in Stoplight Elements (you've got name=something in the example CURL section when provided {"name":"something"} for filter parameter). Yes, you could've used deepObject for style in this specific case, but not all existing tools will serialize it correctly even with deepObject! For example, Stoplight Elements which Scramble uses by default will not properly serialize it.

With this PR, Scramble will document such query parameters like this:

{"name": "filter[name]", "in": "query", "schema": {"type": "string"}}

This makes documentation clear and Try it out section work properly in all OpenAPI tools. Also this is how Scramble PRO documents filters for Spatie Query Builder package.

This also fixes #879 by not keeping duplicated query parameters.

2. Array of scalars (fixes #767)

This is the case when you have a query parameter defined by the following rules:

$request->validate([
    'tags.*' => ['string']
]);

tags parameter now can be passed via this query string:

?tags[]=foo&tags[]=bar

Previously Scramble documented such parameter as tags parameter which is an array of string:

{"name": "tags", "in": "query", "schema": {"type": "array", "items": {"type": "string"}}}

Again, this wasn't properly handled by existing OpenAPI tooling. Not cool! So now Scramble documents it in a way that is understandable by tools (meaning that they produce a correct query string):

{"name": "tags[]", "in": "query", "schema": {"type": "array", "items": {"type": "string"}}}

As you can see, Scramble adds [] suffix to the parameter name. You can see the discussion about this in this issue's comments #767.

What is not handled in this PR

Even more complex structures for query parameters, such as arrays of objects, etc., are left intact (almost). There is simply no good way to document them for the existing OpenAPI tooling.

Scramble, however, marks them with "x-deepObject-style": "qs" ("qs" because qs library serializes deep objects in the way that is understood by PHP). It is not used by any tools out there but it can be used either by tools, or by a developer who uses Scramble.

In operation transformer you can iterate over parameters and if any parameter has this extension set to qs you can do whatever you want. For example, you can add a description helping your users to understand how to properly pass the parameter. Or you can flatten it. Let me know if you do something cool with it!

🥴

[darkbasic]

This is a HUGE BC for us and breaks the whole FE library that is supposed to validate filter typings.

[romalytvynenko]

@darkbasic can you tell me more about your setup? I will look for solutions

[darkbasic]

@romalytvynenko sure. I built a (soon to be open sourced) UI library that extends Vuetify to make it extremely type safe, end to end. We use scramble to generate the openapi documentation, openapi-typescript to generate the typings and openapi-fetch to make the api calls. When we define the headers of a vuetify VDataTableServer we also define if a certain column is filterable or not and our library automatically generates the queries at runtime accordingly, while also statically analysing if the typings are compatible.

Our query filters look like this:

     "v1.insights.index": {
         parameters: {
             query?: {
                vat_number?: {
                    eq?: string | null;
                    contains?: string | null;
                    in?: string | null;
                };

that means that if the header for a column keyed vat_number has filterabled: true we check at compile time if the typings for the corresponding endpoint contain the previous snippet. That way we can know for sure that be and fe are always synced.

Since v0.12.24 scramble generates the following instead:

     "v1.insights.index": {
         parameters: {
             query?: {
                "vat_number[eq]"?: string | null;
                "vat_number[contains]"?: string | null;
                "vat_number[in][]"?: string[];

That won't match the structure that the UI library expects (our "string" filters expect an object with the keys eq, contains and in, while "number" filters also expect gt, gte, lt, lte, etc..).

We encountered issues with the openapi-fetch query param encoder, in fact we used to use a slightly different format for the filters that hit the nesting limit of the encoder:

            filters?: {
                requested_by?: {
                    eq?: string | null;
                    in?: string | null;
                    contains?: string | null;
                };

We initially wrote a custom encoder but since removing the filters prefix was enough to get it working we decided to simply strip the filters prefix instead.

Your breaking change has a particularly unfortunate timing because OpenAPI 3.2 has recently introduced in: querystring that should hopefully help to fix the problem.

I suggest to add a config parameter in scramble that allows to restore the old behaviour, at least until support for 3.2 is out and the path forward is more clear. At the moment we unfortunately had to pin v0.12.23 for the time being.

[romalytvynenko]

@darkbasic thanks for the detailed explanation! I will add the config option. I'm sorry for the inconvenience this change caused for you.

[darkbasic]

Hi @romalytvynenko , any news on this?

[romalytvynenko]

@darkbasic hey
Still planning to do this as soon as possible. Now finishing with vendor types support and I'll make sure to release this fix

[romalytvynenko]

@darkbasic
https://github.com/dedoc/scramble/releases/tag/v0.12.36

You can now set scramble.flatten_deep_query_parameters to false to preserve the old behavior.

scramble/config/scramble.php

Line 119 in fb99275

'flatten_deep_query_parameters' => true,

Let me know if this helped.

[darkbasic]

@romalytvynenko I'm sorry for the late reply but we had some issues with the new param and we didn't manage to debug them until now.

This is how we instantiate scramble. 'flatten_deep_query_parameters' -> false doesn't work there but if we instead modify the default value in the package itself it does. We ended up overriding the package defaults for the moment until you manage to get it working at instantiation time. Thanks!