feat: add HarvestService for API harvesting and OAS conversion

[pasibun]

• Introduced HarvestService to manage API harvesting from a specified endpoint.
• Implemented methods for fetching index URLs, posting API data, and obtaining access tokens.
• Added utility functions for URL building, error handling, and data normalization.

refactor: update OasConversionService to utilize new conversion libraries

• Replaced custom schema conversion logic with @apiture/openapi-down-convert and @scalar/openapi-upgrader.
• Simplified conversion logic for OpenAPI specifications between versions 3.0 and 3.1.
• Enhanced error handling for conversion processes.

test: add unit tests for OasConversionService

• Created tests for converting OpenAPI specifications between versions 3.0 and 3.1.
• Verified preservation of key OpenAPI features during conversion.
• Ensured correct handling of YAML and JSON formats in conversion responses.

chore: update package dependencies

• Updated @redocly/cli to version 2.20.5.
• Added @apiture/openapi-down-convert and @scalar/openapi-upgrader as dependencies.
• Removed unused dependencies from package.json.

[github-actions]

Super-linter summary

Language	Validation result
JSON	Pass ✅
OPENAPI	Pass ✅

All files and directories linted successfully

For more information, see the GitHub Actions workflow run

Powered by Super-linter

[github-actions]

Super-linter summary

Language	Validation result
JSON	Pass ✅
OPENAPI	Pass ✅

All files and directories linted successfully

For more information, see the GitHub Actions workflow run

Powered by Super-linter

[Copilot]

Pull request overview

This PR introduces an automated “harvest” flow for registering OpenAPI sources, and refactors OpenAPI 3.0/3.1 conversion to use dedicated upgrader/down-converter libraries, with new unit tests and dependency/CI housekeeping.

Changes:

• Added HarvestService + HarvestJob and wired a PDOK harvest scheduler into server startup.
• Refactored OasConversionService to use @scalar/openapi-upgrader (3.0→3.1) and @apiture/openapi-down-convert (3.1→3.0), plus added Node’s built-in test runner tests.
• Updated npm dependencies and introduced JSON/OpenAPI lint workflow + Dependabot config.

Reviewed changes

Copilot reviewed 9 out of 10 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
test/OasConversionService.test.js	Adds unit tests validating 3.0↔3.1 conversion behavior and response formatting (JSON/YAML).
services/OasConversionService.js	Switches conversion implementation to new external libraries and adjusts version/serialization behavior.
services/HarvestService.js	New service to fetch index URLs, derive OAS URLs, get tokens, and POST registrations with retry/timeout helpers.
jobs/HarvestJob.js	New scheduler to periodically run harvesting (and on startup), with env-based PDOK setup.
index.js	Loads local .env when supported and starts the harvest scheduler on server launch.
expressServer.js	Removes cookie-parser usage from middleware setup.
package.json	Adds test script, updates/adds conversion dependencies, removes unused deps.
package-lock.json	Locks updated dependency graph to match new/removed packages.
.github/workflows/json-ci.yml	Adds a JSON/OpenAPI lint workflow using Super-Linter.
.github/dependabot.yml	Adds Dependabot config for npm and GitHub Actions updates.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

[github-actions]

Super-linter summary

Language	Validation result
JSON	Pass ✅
OPENAPI	Pass ✅

All files and directories linted successfully

For more information, see the GitHub Actions workflow run

Powered by Super-linter

[github-actions]

Super-linter summary

Language	Validation result
JSON	Pass ✅
OPENAPI	Pass ✅

All files and directories linted successfully

For more information, see the GitHub Actions workflow run

Powered by Super-linter

[github-actions]

Super-linter summary

Language	Validation result
JSON	Pass ✅
OPENAPI	Pass ✅

All files and directories linted successfully

For more information, see the GitHub Actions workflow run

Powered by Super-linter

[github-actions]

Super-linter summary

Language	Validation result
JSON	Pass ✅
OPENAPI	Pass ✅

All files and directories linted successfully

For more information, see the GitHub Actions workflow run

Powered by Super-linter