Skip to content

Commit 9db544c

Browse files
Pijukatelclaudevdusek
authored
docs(openapi): Autofix OpenAPI spec validation errors (#2403)
## Summary Autogenerated OpenAPI fixes suggestions based on validation errors generated from running API integration tests with OpenAPI validator turned on. **Error log:** https://apify-pr-test-env-logs.s3.us-east-1.amazonaws.com/apify/apify-core/26280/api-e8976db3b5b2476d2f01af1a84d4268fc61e839f.log **apify-core version:** https://github.com/apify/apify-core/tree/e8976db3b5b2476d2f01af1a84d4268fc61e839f **Stop reason:** No new fixes possible. All remaining validation errors are deliberate negative tests, known false positives (nullable date-time fields), validator limitations (oneOf for KV store records), missing endpoints (out of scope), or global middleware artifacts (method query parameter override). 16 unique error types fixed out of 20 target. ## Detailed changes description ### Error fixes #### 1. Fix pricingInfo oneOf validation errors - **Files:** `components/schemas/actor-pricing-info/FreeActorPricingInfo.yaml`, `PayPerEventActorPricingInfo.yaml`, `PricePerDatasetItemActorPricingInfo.yaml`, `FlatPricePerMonthActorPricingInfo.yaml` - **Error:** `must have required property 'pricePerUnitUsd'` / `must have required property 'pricingPerEvent'` / `must have required property 'trialMinutes'` / `must have required property 'unitName'` at GET/PUT `/v2/acts/{actorId}` - **Root cause:** The pricingInfo oneOf schemas had mismatched required fields. Each pricing model type had required fields that don't exist for that model (e.g., FREE model doesn't have pricePerUnitUsd). Inlined the pricingModel enum values to each schema so the validator can discriminate correctly. - **Reference:** https://github.com/apify/apify-core/tree/e8976db3b5b2476d2f01af1a84d4268fc61e839f/src/packages/actor/src/pricing/pricing.both.ts#L22 #### 2. Fix maxItems type in RunOptions - **Files:** `components/schemas/actor-runs/RunOptions.yaml:26` - **Error:** `must be integer` at response path `/response/data/options/maxItems` - **Root cause:** maxItems was typed as `integer` but the API returns `null` when not set. Changed to `type: [integer, "null"]`. - **Reference:** https://github.com/apify/apify-core/tree/e8976db3b5b2476d2f01af1a84d4268fc61e839f/src/packages/schemas/src/actor_tasks.ts#L53 #### 3. Fix version DELETE 404 error type - **Files:** `paths/actors/acts@{actorId}@versions@{versionNumber}.yaml:72` - **Error:** `must be equal to one of the allowed values: record-not-found` at DELETE `/v2/acts/{actorId}/versions/{versionNumber}` - **Root cause:** The 404 error response referenced `ActorNotFoundError` but the API returns `RecordOrTokenNotFoundError` when deleting a non-existent version. - **Reference:** https://github.com/apify/apify-core/tree/e8976db3b5b2476d2f01af1a84d4268fc61e839f/src/api/src/routes/actors/versions.ts#L118 #### 4. Fix sourceFiles required fields - **Files:** `components/schemas/actors/SourceCodeFile.yaml` - **Error:** `must have required property 'folder'` / `must have required property 'format'` at response path `/response/data/versions/0/sourceFiles/0` - **Root cause:** `format` and `folder` were marked as required in the schema but are optional fields in the API response - they can be missing from individual source files. - **Reference:** https://github.com/apify/apify-core/tree/e8976db3b5b2476d2f01af1a84d4268fc61e839f/src/packages/schemas/src/actors.ts#L184 #### 5. Fix RequestUrl format:uri on SharedProperties - **Files:** `components/schemas/request-queues/SharedProperties.yaml` - **Error:** `must match format "uri"` at response path for request URL field - **Root cause:** The `format: uri` constraint was too strict - the API accepts and returns URLs that may not pass strict URI validation (e.g., URLs with special characters). Removed `format: uri`. - **Reference:** https://github.com/apify/apify-core/tree/e8976db3b5b2476d2f01af1a84d4268fc61e839f/src/packages/schemas/src/request_queues.ts#L15 #### 6. Fix unsupported media type errors on run endpoints - **Files:** `paths/actors/acts@{actorId}@runs.yaml`, `acts@{actorId}@run-sync.yaml`, `acts@{actorId}@run-sync-get-dataset-items.yaml`, `paths/actor-tasks/actor-tasks@{actorTaskId}@runs.yaml`, `actor-tasks@{actorTaskId}@run-sync.yaml`, `actor-tasks@{actorTaskId}@run-sync-get-dataset-items.yaml` - **Error:** `unsupported media type undefined` at POST on all 6 run endpoints - **Root cause:** The run endpoints use `BODY_PARSER_TYPES.raw` with `type: () => true` which accepts ANY content type. The spec only declared `application/json` content type. Added `"*/*": schema: {}` wildcard to requestBody content. - **Reference:** https://github.com/apify/apify-core/tree/e8976db3b5b2476d2f01af1a84d4268fc61e839f/src/api/src/middleware/body_parser.ts#L61 #### 7. Fix CreateTaskRequest required name field - **Files:** `components/schemas/actor-tasks/CreateTaskRequest.yaml` - **Error:** `must have required property 'name'` at POST `/v2/actor-tasks` - **Root cause:** The `name` field was marked as required, but the API auto-generates a name when not provided. Only `actId` is truly required. - **Reference:** https://github.com/apify/apify-core/tree/e8976db3b5b2476d2f01af1a84d4268fc61e839f/src/packages/actor-server/src/actor_tasks/actor_tasks.server.ts#L417 #### 8. Fix missing waitForFinish parameter on actors runs/last - **Files:** `paths/actors/acts@{actorId}@Runs@last.yaml:63` - **Error:** `Unknown query parameter 'waitForFinish'` at GET `/v2/acts/{actorId}/runs/last` - **Root cause:** The runs/last endpoint forwards all query parameters (including waitForFinish) to the actual run endpoint via `routeToLastRunRoutes()`, but the spec was missing the `waitForFinishRun` parameter declaration. - **Reference:** https://github.com/apify/apify-core/tree/e8976db3b5b2476d2f01af1a84d4268fc61e839f/src/api/src/lib/controllers.ts#L206 #### 9. Fix webhook requestUrl format:uri - **Files:** `components/schemas/webhooks/WebhookCreate.yaml` - **Error:** `must match format "uri"` at POST `/v2/webhooks` - **Root cause:** The API uses a custom `WEBHOOK_REQUEST_URL_REGEXP` that accepts URLs not matching strict `format: uri` validation. Removed `format: uri`. - **Reference:** https://github.com/apify/apify-core/tree/e8976db3b5b2476d2f01af1a84d4268fc61e839f/src/packages/consts/src/webhooks.ts#L31 #### 10. Fix request queue payload type - **Files:** `components/schemas/request-queues/RequestBase.yaml` - **Error:** `must be object,null` at POST `/v2/request-queues/{queueId}/requests` - **Root cause:** The payload field accepted only `[object, "null"]` but the API also accepts string payloads (String|Buffer|Object). Changed to `[string, object, "null"]`. - **Reference:** https://github.com/apify/apify-core/tree/e8976db3b5b2476d2f01af1a84d4268fc61e839f/src/packages/schemas/src/request_queues.ts#L40 #### 11. Fix status query parameter serialization - **Files:** `components/parameters/runAndBuildParameters.yaml` (status parameter) - **Error:** `Parameter 'status' must be url encoded. Its value may not contain reserved characters.` at GET `/v2/acts/{actorId}/runs?status=SUCCEEDED,FAILED` - **Root cause:** The `status` parameter accepts comma-separated values parsed by `parseCommaSeparatedString()`. Changed to `type: array` with `items: {type: string}` and `explode: false` for proper comma-separated serialization. - **Reference:** https://github.com/apify/apify-core/tree/e8976db3b5b2476d2f01af1a84d4268fc61e839f/src/api/src/routes/actors/run_list.ts#L117 #### 12. Fix filter query parameter serialization - **Files:** `paths/request-queues/request-queues@{queueId}@requests.yaml` (filter parameter) - **Error:** `Parameter 'filter' must be url encoded. Its value may not contain reserved characters.` at GET `/v2/request-queues/{queueId}/requests?filter=locked,pending` - **Root cause:** The `filter` parameter accepts comma-separated values parsed by `filterString?.split(',')`. Changed to `type: array` with `items: {type: string, enum: [locked, pending]}` and `explode: false`. - **Reference:** https://github.com/apify/apify-core/tree/e8976db3b5b2476d2f01af1a84d4268fc61e839f/src/api/src/routes/request_queues/requests.ts#L386 #### 13. Fix startedAfter/startedBefore reserved character handling - **Files:** `components/parameters/runAndBuildParameters.yaml` (startedAfter & startedBefore) - **Error:** `Parameter 'startedAfter' must be url encoded. Its value may not contain reserved characters.` / same for `startedBefore` - **Root cause:** ISO 8601 date-time values contain colons which are RFC 3986 reserved characters. Added `allowReserved: true` to both parameters. - **Reference:** https://github.com/apify/apify-core/tree/e8976db3b5b2476d2f01af1a84d4268fc61e839f/src/api/src/routes/actors/run_list.ts#L105 #### 14. Fix missing waitForFinish parameter on actor-tasks runs/last - **Files:** `paths/actor-tasks/actor-tasks@{actorTaskId}@Runs@last.yaml:55` - **Error:** `Unknown query parameter 'waitForFinish'` at GET `/v2/actor-tasks/{actorTaskId}/runs/last` - **Root cause:** Same as fix #8 but for actor-tasks. The actor-tasks runs/last endpoint uses the same `routeToLastRunRoutes()` function, forwarding all query parameters including waitForFinish. - **Reference:** https://github.com/apify/apify-core/tree/e8976db3b5b2476d2f01af1a84d4268fc61e839f/src/api/src/routes/actor_tasks/last_run.ts#L39 #### 15. Fix Content-Encoding enum missing identity value - **Files:** `components/objects/key-value-stores/key-value-stores@{storeId}@records@{recordKey}put.yaml:13` - **Error:** `must be equal to one of the allowed values: gzip, deflate, br` at PUT `/v2/key-value-stores/{storeId}/records/{recordKey}` - **Root cause:** The API accepts `identity` encoding for HTML records, checked at `record.ts:102` against `[...RECORD_SUPPORTED_ENCODING_TYPES_LIST, IDENTITY_ENCODING_TYPE]`. The `identity` value was missing from the spec's Content-Encoding enum. - **Reference:** https://github.com/apify/apify-core/tree/e8976db3b5b2476d2f01af1a84d4268fc61e839f/src/api/src/routes/key_value_stores/record.ts#L102 #### 16. Fix missing view parameter on run-sync-get-dataset-items endpoints - **Files:** `paths/actors/acts@{actorId}@run-sync-get-dataset-items.yaml`, `paths/actor-tasks/actor-tasks@{actorTaskId}@run-sync-get-dataset-items.yaml`, `components/parameters/datasetItemsParameters.yaml` - **Error:** `Unknown query parameter 'view'` at POST `/v2/acts/{actorId}/run-sync-get-dataset-items` and POST `/v2/actor-tasks/{actorTaskId}/run-sync-get-dataset-items` - **Root cause:** The run-sync-get-dataset-items endpoints call `respondWithDatasetItemsStream()` which processes the `view` query parameter (at `datasets.ts:328`), but the spec was missing this parameter. Also extracted the inline `view` parameter from `datasets@{datasetId}@items.yaml` into a reusable component in `datasetItemsParameters.yaml`. - **Reference:** https://github.com/apify/apify-core/tree/e8976db3b5b2476d2f01af1a84d4268fc61e839f/src/api/src/routes/actors/run_sync_dataset.ts#L57 ### Refactoring #### Extract view parameter to reusable component - **Files:** `components/parameters/datasetItemsParameters.yaml`, `paths/datasets/datasets@{datasetId}@items.yaml` - Moved the inline `view` parameter definition from the dataset items endpoint into the shared `datasetItemsParameters.yaml` file. This follows the apify-docs guideline to prefer `$ref` reuse over duplication, and enables the fix for run-sync-get-dataset-items endpoints. #### Replace single-item enums with const - **Files:** `components/schemas/actor-pricing-info/PayPerEventActorPricingInfo.yaml:11`, `FreeActorPricingInfo.yaml:10`, `PricePerDatasetItemActorPricingInfo.yaml:12`, `FlatPricePerMonthActorPricingInfo.yaml:12`, `paths/request-queues/request-queues@{queueId}@requests@batch.yaml:95` - Per reviewer feedback and updated AGENTS.md syntax hints, replaced all single-item `enum` definitions with `const` (OpenAPI 3.1 syntax). Affects 4 pricingModel discriminator fields and 1 Content-Type header parameter. #### AGENTS.md syntax hints update - **Files:** `AGENTS.md` - Maintainer commit (`cb46377`) adding `const` syntax hints to the OpenAPI specification changes section. This motivated the enum-to-const refactoring above. ## Unfixed errors ### False positives #### Nullable date-time fields in taggedBuilds - **Error:** `must be null` / `must match a schema in anyOf` at response path `/response/data/taggedBuilds/latest` on GET/PUT `/v2/acts/{actorId}` (~193 occurrences) - **Root cause:** The `TaggedBuildInfo.yaml` schema has `finishedAt: type: [string, "null"], format: date-time`. The express-openapi-validator incorrectly rejects null values for date-time formatted fields. This is a known validator limitation. - **Reference:** https://github.com/apify/apify-core/tree/e8976db3b5b2476d2f01af1a84d4268fc61e839f/src/packages/types/src/actor.ts #### Nullable date-time fields in lastDispatch - **Error:** `must be null` / `must match a schema in anyOf` at response path `/response/data/lastDispatch` on PUT `/v2/webhooks/{webhookId}` (~9 occurrences) - **Root cause:** Same nullable date-time false positive. `ExampleWebhookDispatch.yaml` has `finishedAt: type: [string, "null"], format: date-time`. - **Reference:** https://github.com/apify/apify-core/tree/e8976db3b5b2476d2f01af1a84d4268fc61e839f/src/packages/integrations-server/src/webhooks/webhook_dispatch_sender.ts#L267 #### Handler mutation of input field (anyOf) - **Error:** `must be object` / `must be null` / `must match a schema in anyOf` at request path `/body/input` on POST `/v2/actor-tasks` and POST `/v2/schedules` (~8 occurrences) - **Root cause:** The handler calls `JSON.stringify(req.body.input)` before the OpenAPI validator runs (validator is in monitoring-only mode via `res.on('finish', ...)`), turning the object into a string. The validator then sees a string where it expects object|null. - **Reference:** https://github.com/apify/apify-core/tree/e8976db3b5b2476d2f01af1a84d4268fc61e839f/src/api/src/routes/actor_tasks/tasks.ts #### KV store record oneOf response validation - **Error:** `must match exactly one schema in oneOf` at response path `/response` on GET `/v2/key-value-stores/{storeId}/records/{recordKey}` (~341 occurrences) - **Root cause:** The endpoint returns various content types and the validator cannot properly evaluate oneOf across different media types in the response schema. The spec is correct. - **Reference:** https://github.com/apify/apify-core/tree/e8976db3b5b2476d2f01af1a84d4268fc61e839f/src/api/src/routes/key_value_stores/record.ts ### Out of scope errors #### Missing actor-runs sub-route endpoints - **Error:** `not found` at GET `/v2/actor-runs/{runId}/log`, `/v2/actor-runs/{runId}/dataset`, `/v2/actor-runs/{runId}/key-value-store`, `/v2/actor-runs/{runId}/request-queue` (~277 occurrences) - **Root cause:** These endpoints exist in the API but are not defined in the OpenAPI spec. Adding new endpoints is out of scope for this PR. #### Missing validate-input endpoint - **Error:** `not found` at POST `/v2/acts/{actorId}/validate-input` (~46 occurrences) - **Root cause:** This endpoint exists in the API but is not defined in the OpenAPI spec. Adding new endpoints is out of scope. #### Global method query parameter override - **Error:** `Unknown query parameter 'method'` on various endpoints (~4 occurrences) - **Root cause:** The API has a global middleware (`overrideMethodMiddleware`) that accepts `?method=` on every endpoint to override the HTTP method. This is a cross-cutting transport-level concern, not a per-endpoint parameter. Not appropriate to add to every endpoint in the spec. ### Potential API bugs _None identified._ ## Issues Partially implements: #2286 --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Vlada Dusek <v.dusek96@gmail.com>
1 parent 8dae887 commit 9db544c

27 files changed

Lines changed: 75 additions & 48 deletions

AGENTS.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,22 @@ Add code samples by creating files in `apify-api/openapi/code_samples/{javascrip
7171
- Each endpoint that has `runs/last` in its path or that has any ID related parameter (for example `actorId`, `buildId`, `runId`, `datasetId` and so on) should have at least one 404 (Not Found) error.
7272
- Each endpoint that has `requestBody` should have at least following error responses: 413 (Payload Too Large), 415 (Unsupported Media Type).
7373

74+
#### Syntax hints
75+
- Instead of using one item `enum`, use `const`:
76+
Avoid this:
77+
```yaml
78+
schema:
79+
type: string
80+
enum:
81+
- "constantValue"
82+
```
83+
Use this:
84+
```yaml
85+
schema:
86+
type: string
87+
const: "constantValue"
88+
```
89+
7490
### Theme system
7591
7692
Uses `@apify/docs-theme` package - a shared theme across all 6+ documentation repos. Don't modify theme files directly. Changes to the theme propagate via CI to all projects.

apify-api/openapi/components/objects/key-value-stores/key-value-stores@{storeId}@records@{recordKey}put.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ shared: &shared
1414
- gzip
1515
- deflate
1616
- br
17+
- identity
1718
type: string
1819
requestBody:
1920
description: ""

apify-api/openapi/components/parameters/datasetItemsParameters.yaml

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -206,6 +206,17 @@ descDataset:
206206
type: boolean
207207
example: true
208208

209+
view:
210+
name: view
211+
in: query
212+
description: |
213+
Defines the view configuration for dataset items based on the schema definition.
214+
This parameter determines how the data will be filtered and presented.
215+
For complete specification details, see the [dataset schema documentation](/platform/actors/development/actor-definition/dataset-schema).
216+
schema:
217+
type: string
218+
example: overview
219+
209220
skipFailedPages:
210221
name: skipFailedPages
211222
in: query

apify-api/openapi/components/parameters/runAndBuildParameters.yaml

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -258,10 +258,12 @@ status:
258258
Single status or comma-separated list of statuses, see ([available
259259
statuses](https://docs.apify.com/platform/actors/running/runs-and-builds#lifecycle)). Used to filter runs by the specified statuses only.
260260
style: form
261-
explode: true
261+
explode: false
262262
schema:
263-
type: string
264-
example: SUCCEEDED
263+
type: array
264+
items:
265+
type: string
266+
example: [SUCCEEDED]
265267

266268
startedAfter:
267269
name: startedAfter
@@ -271,6 +273,7 @@ startedAfter:
271273
The value must be a valid ISO 8601 datetime string (UTC).
272274
style: form
273275
explode: true
276+
allowReserved: true
274277
schema:
275278
type: string
276279
format: date-time
@@ -284,6 +287,7 @@ startedBefore:
284287
The value must be a valid ISO 8601 datetime string (UTC).
285288
style: form
286289
explode: true
290+
allowReserved: true
287291
schema:
288292
type: string
289293
format: date-time

apify-api/openapi/components/schemas/actor-pricing-info/FlatPricePerMonthActorPricingInfo.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,8 @@ allOf:
88
- trialMinutes
99
properties:
1010
pricingModel:
11-
$ref: ./PricingModel.yaml
11+
type: string
12+
const: FLAT_PRICE_PER_MONTH
1213
trialMinutes:
1314
type: integer
1415
description: For how long this Actor can be used for free in trial period

apify-api/openapi/components/schemas/actor-pricing-info/FreeActorPricingInfo.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,4 +6,5 @@ allOf:
66
- pricingModel
77
properties:
88
pricingModel:
9-
$ref: ./PricingModel.yaml
9+
type: string
10+
const: FREE

apify-api/openapi/components/schemas/actor-pricing-info/PayPerEventActorPricingInfo.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,8 @@ allOf:
77
- pricingPerEvent
88
properties:
99
pricingModel:
10-
$ref: ./PricingModel.yaml
10+
type: string
11+
const: PAY_PER_EVENT
1112
pricingPerEvent:
1213
type: object
1314
properties:

apify-api/openapi/components/schemas/actor-pricing-info/PricePerDatasetItemActorPricingInfo.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,8 @@ allOf:
88
- unitName
99
properties:
1010
pricingModel:
11-
$ref: ./PricingModel.yaml
11+
type: string
12+
const: PRICE_PER_DATASET_ITEM
1213
unitName:
1314
type: string
1415
description: Name of the unit that is being charged

apify-api/openapi/components/schemas/actor-pricing-info/PricingModel.yaml

Lines changed: 0 additions & 7 deletions
This file was deleted.

apify-api/openapi/components/schemas/actor-runs/RunOptions.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ properties:
2323
minimum: 0
2424
examples: [2048]
2525
maxItems:
26-
type: integer
26+
type: [integer, "null"]
2727
minimum: 1
2828
examples: [1000]
2929
maxTotalChargeUsd:

0 commit comments

Comments
 (0)