[SPARK-56998] Add SECURITY.md + AGENTS.md Security section for scan-agent discoverability

**This is a proposal for the PMC to review — please correct, reject, or discuss as needed.** Nothing here is a requirement; the maintainer is the decision-maker.

This adds a `SECURITY.md` to the repo root and a `Security` section to the existing `AGENTS.md` so an automated scan agent can mechanically discover the project's security model via the conventional `AGENTS.md → SECURITY.md → model URL` chain. The chain terminates at the existing <https://spark.apache.org/docs/latest/security.html> page — nothing about the model content itself changes.

Context: the ASF Security team is preparing the project for an automated agentic security scan we're piloting. Such scans refuse to run if the model isn't discoverable by that path (refusing upfront beats wasting PMC reviewer cycles on a noise-heavy run against an unknown model). Discoverability is the one hard gate; everything else is suggestion. The Security team has reached out separately on the PMC's private list with the program details; this PR is the public-facing repo piece.

The Security team uses [`threat-model-producer`](https://gist.github.com/potiuk/da14a826283038ddfe38cc9fe6310573) as the rubric for what a complete model looks like — but this PR is just the *link*; the existing `security.html` content is accepted as the model.

After this lands on `master`, the same two files would need to be on `branch-3.5` for the second scan target — happy to open a cherry-pick PR for that, or leave it to the PMC.

Questions / pushback welcome. Happy to adjust the wording or move the section if the project has a house style.

Closes #55933 from potiuk/asf-security/discoverability-2026-05-18.

Lead-authored-by: Jarek Potiuk <jarek@potiuk.com>
Co-authored-by: Xiao Li <gatorsmile@gmail.com>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
(cherry picked from commit 411dedc)
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>

Author: 2 people

AGENTS.md

@@ -128,3 +128,9 @@ DO NOT push to the upstream repo. Always push to the personal fork. Open PRs aga
 DO NOT force push or use `--amend` on pushed commits unless the user explicitly asks. If the remote branch has new commits, fetch and rebase before pushing.
 
 Always get user approval before external operations such as pushing commits, creating PRs, or posting comments. Use `gh pr create` to open PRs. If `gh` is not installed, generate the GitHub PR URL for the user and recommend installing the GitHub CLI.
+
+## Security
+
+Security model: [SECURITY.md](./SECURITY.md)
+
+Agents that scan this repository should consult `SECURITY.md` for the project's threat model, in-scope / out-of-scope declarations, and known non-findings before reporting issues.

SECURITY.md

@@ -0,0 +1,13 @@
+# Security policy
+
+## Reporting a vulnerability
+
+To report a security vulnerability in Apache Spark, follow the ASF security process at <https://www.apache.org/security/>. Reports can be sent privately to security@apache.org or to private@spark.apache.org.
+
+## Threat model
+
+Apache Spark's security guidance for operators — covering authentication, encryption, access control, Kerberos integration, trust boundaries, environmental assumptions, security properties provided and not provided, and downstream operator responsibilities — is documented at:
+
+<https://spark.apache.org/docs/latest/security.html>
+
+Automated security scanning agents should consult that document for the project's in-scope / out-of-scope declarations before reporting issues.