Merge pull request #896 from pfrest/next_minor

v2.8.0 Features & Fixes

Author: jaredhendrickson13

.github/dependabot.yml

@@ -11,16 +11,31 @@ updates:
     target-branch: "master"
     schedule:
       interval: "monthly"
+    groups:
+      composer:
+        patterns:
+          - "*"
+
   - package-ecosystem: "pip"
     directory: "/"
     target-branch: "master"
     schedule:
       interval: "monthly"
+    groups:
+      pip:
+        patterns:
+          - "*"
+
   - package-ecosystem: "github-actions"
     directory: "/"
     target-branch: "master"
     schedule:
       interval: "monthly"
+    groups:
+      github-actions:
+        patterns:
+          - "*"
+
   - package-ecosystem: "npm"
     directory: "/"
     target-branch: "master"
@@ -29,20 +44,7 @@ updates:
       interval: "monthly"
     commit-message:
       prefix: "chore"
-
-  # LEGACY BRANCH (v1)
-  - package-ecosystem: "composer"
-    directory: "/"
-    target-branch: "legacy"
-    schedule:
-      interval: "monthly"
-  - package-ecosystem: "pip"
-    directory: "/"
-    target-branch: "legacy"
-    schedule:
-      interval: "monthly"
-  - package-ecosystem: "github-actions"
-    directory: "/"
-    target-branch: "legacy"
-    schedule:
-      interval: "monthly"
+    groups:
+      npm:
+        patterns:
+          - "*"

.github/instructions/README.md

@@ -0,0 +1,34 @@
+---
+applyTo: ".github/instructions/**/*.md"
+---
+
+# About `.github/instructions/`
+
+This directory contains **GitHub Copilot path-scoped instructions**. Each `*.instructions.md` file declares an `applyTo` glob in its YAML frontmatter; Copilot automatically loads the matching file when you edit a file under that path.
+
+## Files
+
+| File                          | `applyTo` glob                                             | Purpose                                                   |
+| ----------------------------- | ---------------------------------------------------------- | --------------------------------------------------------- |
+| `php.instructions.md`         | `pfSense-pkg-RESTAPI/files/usr/local/pkg/RESTAPI/**/*.inc` | Global PHP rules for all package source files.            |
+| `models.instructions.md`      | `.../RESTAPI/Models/**/*.inc`                              | Model class authoring rules.                              |
+| `endpoints.instructions.md`   | `.../RESTAPI/Endpoints/**/*.inc`                           | Endpoint class authoring rules.                           |
+| `validators.instructions.md`  | `.../RESTAPI/Validators/**/*.inc`                          | Reusable Validator authoring rules.                       |
+| `dispatchers.instructions.md` | `.../RESTAPI/Dispatchers/**/*.inc`                         | Background Dispatcher authoring rules.                    |
+| `tests.instructions.md`       | `.../RESTAPI/Tests/**/*.inc`                               | TestCase authoring rules (custom framework, not PHPUnit). |
+
+## How to extend
+
+1. Copy an existing `*.instructions.md` and edit the `applyTo` glob.
+2. Keep each file focused on one path scope. Don't duplicate `AGENTS.md` — link to it.
+3. Keep the rules short and prescriptive. Long-form examples belong in `.github/skills/` or `docs/`.
+4. After editing, verify the `applyTo` glob actually matches the file you expect — Copilot only loads the file when the glob matches.
+
+## Authoritative sources
+
+These files are guard-rails. The full picture is:
+
+- `AGENTS.md` (root) — architecture and rules.
+- `.github/skills/` — task playbooks with concrete examples.
+- `docs/CONTRIBUTING.md` and `docs/BUILDING_CUSTOM_*.md` — human reference docs.
+- `https://pfrest.org/php-docs/` — generated PHPDoc.

.github/instructions/dispatchers.instructions.md

@@ -0,0 +1,72 @@
+---
+applyTo: "pfSense-pkg-RESTAPI/files/usr/local/pkg/RESTAPI/Dispatchers/**/*.inc"
+---
+
+# Dispatchers — `RESTAPI\Dispatchers\*`
+
+Dispatchers run work in the background so HTTP requests stay responsive. Use a Dispatcher for any operation that can take more than ~1 second: filter reloads, service restarts, certificate issuance, HA sync, etc.
+
+For full guidance and worked examples see:
+
+- `.github/skills/validation-command-dispatcher.md`
+- `docs/BUILDING_CUSTOM_DISPATCHER_CLASSES.md`
+
+## Required shape
+
+```php
+<?php
+
+namespace RESTAPI\Dispatchers;
+
+use RESTAPI\Core\Dispatcher;
+
+/**
+ * Defines a Dispatcher that <does the long-running thing>.
+ */
+class <Area>ApplyDispatcher extends Dispatcher {
+    public int $timeout = 300;          # optional override
+    public int $max_queue = 10;         # optional override
+    public string $schedule = '';       # 5-field cron string; set to register a CronJob
+    protected array $required_packages = [];
+    protected array $package_includes = [];
+
+    protected function _process(mixed ...$arguments): void {
+        # Do the actual work here. This runs in a forked PHP process.
+    }
+}
+```
+
+## Conventions
+
+- Class name ends in `Dispatcher` (e.g. `FirewallApplyDispatcher`, `DNSResolverApplyDispatcher`, `RESTAPISettingsSyncDispatcher`).
+- Implement `_process(mixed ...$arguments): void`. Never expose work via a public method other than the inherited `process()`/`spawn_process()`.
+- Honor `$this->async` when the underlying pfSense function offers both sync and async forms (e.g. `filter_configure()` vs `filter_configure_sync()`).
+- For Dispatchers that depend on a pfSense package, list it in `$required_packages` (full `pfSense-pkg-...` name) and any required PHP includes in `$package_includes`. Missing dependencies throw `FailedDependencyError`.
+- Use `$this->schedule = '*/5 * * * *';` to register a `CronJob` automatically (handled by `manage.php` at install time).
+
+## Triggering from a Model
+
+```php
+public function apply(): void {
+    (new <Area>ApplyDispatcher(async: $this->async))->spawn_process();
+}
+```
+
+`spawn_process()` returns a PID, writes a PID file under `/tmp/`, and respects `$max_queue` (overflow throws `ServiceUnavailableError` with `DISPATCHER_TOO_MANY_QUEUED_PROCESSES`).
+
+## Hard rules
+
+- **Never** call `filter_configure*()`, `services_*_configure()`, or other long-running pfSense functions directly from a Model — always wrap them in a Dispatcher.
+- **Never** use bare `exec()` / `shell_exec()` / `passthru()`. Use `RESTAPI\Core\Command`.
+- **Never** rely on `_process()` running synchronously in tests unless you instantiate with `async: false`.
+- The PID file lifecycle is owned by `Core/Dispatcher.inc`. Do not write your own locking.
+
+## Manual invocation (debugging)
+
+On the pfSense host:
+
+```bash
+pfsense-restapi notifydispatcher <DispatcherClassShortName>
+```
+
+This runs the dispatcher synchronously, queues behind any existing instance, and writes `/tmp/<DispatcherName>.lock` while running.

.github/instructions/endpoints.instructions.md

@@ -0,0 +1,71 @@
+---
+applyTo: "pfSense-pkg-RESTAPI/files/usr/local/pkg/RESTAPI/Endpoints/**/*.inc"
+---
+
+# Endpoints — `RESTAPI\Endpoints\*`
+
+Endpoints are **thin URL → Model adapters**. Most are 10–25 lines.
+
+For full guidance and worked examples see:
+
+- `.github/skills/endpoint-model-field.md`
+- `docs/BUILDING_CUSTOM_ENDPOINT_CLASSES.md`
+
+## Required shape
+
+```php
+<?php
+
+namespace RESTAPI\Endpoints;
+
+require_once 'RESTAPI/autoloader.inc';
+
+use RESTAPI\Core\Endpoint;
+
+class <Area><Resource>Endpoint extends Endpoint {
+    public function __construct() {
+        $this->url = '/api/v2/<area>/<resource>';
+        $this->model_name = '<ModelClassName>';
+        $this->request_method_options = [/* GET, POST, PATCH, PUT, DELETE */];
+        # Optional: $this->many, $this->*_help_text, $this->auth_methods, ...
+        parent::__construct();
+    }
+}
+```
+
+## Naming and URLs
+
+- Singular endpoint: `<Area><Resource>Endpoint` → `/api/v2/<area>/<resource>`.
+- Collection endpoint (`many: true`): pluralize, e.g. `FirewallAliasesEndpoint` → `/api/v2/firewall/aliases`.
+- Apply endpoint: `<Area>ApplyEndpoint` → `/api/v2/<area>/apply`.
+- File name = class name + `.inc`.
+
+## Method semantics (enforced in `Core/Endpoint.inc::check_construct`)
+
+| `many`  | Method   | Model method called |
+| ------- | -------- | ------------------- |
+| `false` | `GET`    | `read()`            |
+| `false` | `POST`   | `create()`          |
+| `false` | `PATCH`  | `update()`          |
+| `false` | `DELETE` | `delete()`          |
+| `true`  | `GET`    | `read_all()`        |
+| `true`  | `PUT`    | `replace_all()`     |
+| `true`  | `DELETE` | `delete_many()`     |
+
+Setting `$this->many = true;` while `$model->many` is `false` throws `ENDPOINT_MANY_WITHOUT_MANY_MODEL`.
+
+## Hard rules
+
+- **Never** override `get()`, `post()`, `patch()`, `put()`, or `delete()`. There are zero such overrides in `Endpoints/` today.
+- **Never** put business logic in the Endpoint. All behavior belongs in the Model (or in a Dispatcher invoked from the Model).
+- **Never** hand-roll auth/ACL/privilege handling. Use `requires_auth`, `auth_methods`, `ignore_*` properties.
+- **Never** hard-code privilege names. They are derived from `$url` + method automatically.
+- **Never** edit the generated PHP files in the pfSense webroot. They are produced by `manage.php buildendpoints`.
+
+## Optional Endpoint properties
+
+Set when needed, leave default otherwise: `requires_auth`, `auth_methods`, `ignore_read_only`, `ignore_interfaces`, `ignore_enabled`, `ignore_acl`, `tag` (overrides default OpenAPI tag derived from URL), `*_help_text`, `deprecated`, `limit`, `offset`, `sort_by`, `sort_order`, `sort_flags`, `encode_content_handlers`, `decode_content_handlers`, `resource_link_set`. See `Core/Endpoint.inc` PHPDoc.
+
+## Help text
+
+Set `$this->get_help_text`, `$this->post_help_text`, `$this->patch_help_text`, `$this->put_help_text`, `$this->delete_help_text` only when the auto-generated default (built from the Model's `verbose_name`/`verbose_name_plural`) is misleading.

.github/instructions/models.instructions.md

@@ -0,0 +1,77 @@
+---
+applyTo: "pfSense-pkg-RESTAPI/files/usr/local/pkg/RESTAPI/Models/**/*.inc"
+---
+
+# Models — `RESTAPI\Models\*`
+
+Models hold the actual feature behavior of the package. Endpoints are thin adapters; everything that _does_ something belongs here.
+
+For full guidance and worked examples see:
+
+- `.github/skills/endpoint-model-field.md`
+- `.github/skills/anatomy-of-a-feature.md`
+- `docs/BUILDING_CUSTOM_MODEL_CLASSES.md`
+
+## Constructor signature is fixed
+
+```php
+public function __construct(
+    mixed $id = null,
+    mixed $parent_id = null,
+    mixed $data = [],
+    mixed ...$options,
+) {
+    # 1. Set Model attributes (config_path, many, subsystem, ...)
+    # 2. Define Field objects on $this
+    # 3. parent::__construct(...) MUST be the last statement
+    parent::__construct($id, $parent_id, $data, ...$options);
+}
+```
+
+Order matters. Field objects must exist before the parent constructor runs.
+
+## Naming
+
+- Class: PascalCase, singular noun. `FirewallAlias`, `DNSResolverHostOverrideAlias`, `SystemHostname`.
+- File: `<ClassName>.inc`.
+- One class per file.
+
+## Schema lives on Fields
+
+Define every property as a typed `Field` (`StringField`, `IntegerField`, `BooleanField`, `ForeignModelField`, `NestedModelField`, etc.). Use Field constructor arguments for `required`, `default`, `choices`, `unique`, `sensitive`, `read_only`, `write_only`, `representation_only`, `many`, `delimiter`, `internal_name`, `internal_namespace`, `conditions`, `validators`, `verbose_name`, `help_text`. Do not validate inside the constructor — let Fields and Validators do it.
+
+## Validation
+
+| Need                         | Mechanism                                                              |
+| ---------------------------- | ---------------------------------------------------------------------- |
+| Reusable rule                | A class in `RESTAPI/Validators/` attached via `validators: [...]`.     |
+| Field-specific, not reusable | `validate_<field_name>($value): <type>` returning the validated value. |
+| Cross-field                  | `validate_extra(): void`.                                              |
+
+All three throw `RESTAPI\Responses\ValidationError` with a stable, namespaced `response_id`.
+
+## `apply()` and Dispatchers
+
+If the Model mutates a service (filter, DNS, services, certs, HA), implement:
+
+```php
+public function apply(): void {
+    (new <Area>ApplyDispatcher(async: $this->async))->spawn_process();
+}
+```
+
+Set `$this->always_apply = true;` for singleton settings Models that should apply on every successful update.
+
+Never call `filter_configure*()`, `services_*_configure()`, or other long-running pfSense functions directly from a Model — wrap them in a Dispatcher.
+
+## Non-config-backed Models
+
+If the Model does not store data in `$config`, set `$this->internal_callable = 'method_name';` and provide that method returning an array (single object, or list of arrays for `many` Models). Override `_create()` / `_update()` / `_delete()` only when needed.
+
+## Hard rules
+
+- No bare `exec()` / `shell_exec()` / `passthru()`. Use `RESTAPI\Core\Command`.
+- No pfSense Plus-only functions or paths.
+- Sensitive values must use `sensitive: true` on the Field.
+- Errors throw `RESTAPI\Responses\*` with a stable `UPPER_SNAKE_CASE` `response_id`.
+- Never edit generated files in the pfSense webroot — they are produced by `manage.php buildendpoints` from your Endpoint class.

.github/instructions/php.instructions.md

@@ -0,0 +1,50 @@
+---
+applyTo: "pfSense-pkg-RESTAPI/files/usr/local/pkg/RESTAPI/**/*.inc"
+---
+
+# pfSense-pkg-RESTAPI — global PHP rules
+
+These rules apply to **every** PHP `.inc` file under `pfSense-pkg-RESTAPI/files/usr/local/pkg/RESTAPI/`. Path-specific instructions in sibling files extend these for `Models/`, `Endpoints/`, `Validators/`, `Dispatchers/`, and `Tests/`.
+
+Authoritative reference: `AGENTS.md` and `.github/skills/`.
+
+## File preamble (required, in this order)
+
+```php
+<?php
+
+namespace RESTAPI\<Subnamespace>;
+
+require_once 'RESTAPI/autoloader.inc';
+
+use RESTAPI\Core\<...>;
+```
+
+- File extension is **`.inc`** (not `.php`).
+- File name matches the class name exactly (`SystemHostname.inc` → `class SystemHostname`).
+- Namespace mirrors the directory: `RESTAPI\Models`, `RESTAPI\Endpoints`, `RESTAPI\Validators`, `RESTAPI\Dispatchers`, `RESTAPI\Tests`, etc.
+
+## Conventions
+
+- **PHP 8.2.** Use named arguments at call sites — the codebase consistently does this (`new StringField(required: true, default: '', ...)`).
+- **Comments:** `#` for short inline notes (matches existing code). Use `/** ... */` PHPDoc on every public class and method — PHPDoc rendering is gated by CI.
+- **`response_id` values:** `UPPER_SNAKE_CASE`, prefixed with the class/feature (e.g. `INVALID_FIREWALL_ALIAS_NAME`, `IP_ADDRESS_VALIDATOR_FAILED`).
+- **Throw, don't return errors.** Use a `RESTAPI\Responses\*` class with a stable `response_id`. Never throw bare `\Exception`.
+- **Shell execution:** use `RESTAPI\Core\Command`, never bare `exec()` / `shell_exec()` / `passthru()`.
+- **Sensitive values:** mark Fields `sensitive: true`. Never log secrets.
+
+## pfSense CE only
+
+Never reference symbols, files, or behaviors that exist only in pfSense Plus. If you cannot find it in pfSense CE source, do not use it.
+
+## Formatting
+
+Code must pass:
+
+```bash
+./node_modules/.bin/prettier --check ./pfSense-pkg-RESTAPI/files
+phplint -vvv --no-cache
+./phpdoc
+```
+
+CI enforces all three.