Skip to content

Commit b77e469

Browse files
peter-at-progresspeter-at-progress
committed
Add notify-new-cves action for daily CVE notifications
Implements automated daily notifications for newly discovered CVEs via Microsoft Teams. Features: - Queries scan database for CVEs first observed in the last N days - Groups by product/channel with severity counts - Sends formatted Teams message with CVE links and details - Supports GitHub Actions summary output - Read-only database access with configurable lookback period Action inputs: - database-url: PostgreSQL connection string (read-only) - data-repo-path: Path to chef-vuln-scan-data for metadata - teams-webhook-url: Microsoft Teams incoming webhook - lookback-days: CVE discovery window (default: 1) See SETUP.md for configuration details and workflow integration examples. Signed-off-by: Peter Arsenault <parsenau@progress.com>
1 parent 59f6caf commit b77e469

4 files changed

Lines changed: 1302 additions & 0 deletions

File tree

‎.github/actions/notify-new-cves/README.md‎

Lines changed: 137 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,137 @@
1+
# notify-new-cves Action
2+
3+
Queries the Chef vulnerability analytics database for CVEs first observed within the last 25 hours, enriches each finding with full Grype match details from `chef-vuln-scan-data`, and dispatches notifications to configured channels.
4+
5+
## How It Works
6+
7+
### 1. Detection
8+
Runs a SQL query against `native_cve_details`:
9+
```sql
10+
SELECT * FROM native_cve_details d
11+
LEFT JOIN native_scan_results r ON ...
12+
WHERE d.severity = ANY(severities)
13+
AND d.first_observed_at >= NOW() - INTERVAL '25 hours'
14+
AND d.last_seen_at >= NOW() - INTERVAL '25 hours'
15+
```
16+
17+
The dual filter on `first_observed_at` and `last_seen_at` ensures only truly new CVEs are notified (appeared recently AND still present in the latest scan).
18+
19+
The 25-hour window provides resilience for workflows that run slightly late.
20+
21+
### 2. Enrichment
22+
For each CVE found in the database, the script:
23+
- Globs `chef-vuln-scan-data/{scan_mode}/{product}/{channel}/{download_site}/**/scanners/grype.latest.json`
24+
- Finds the match where `vulnerability.id`, `artifact.name`, and `artifact.version` match the DB row
25+
- Extracts the full `vulnerability`, `relatedVulnerabilities`, `matchDetails`, and `artifact` objects
26+
- Reads `metadata.json` from the same directory for scan provenance (timestamp, Grype version, DB version)
27+
28+
### 3. Notification
29+
Each CVE is formatted as a Microsoft Teams Adaptive Card with:
30+
- **Header**: CVE ID, severity, product, version
31+
- **Details**: CVSS score, EPSS percentile, fix state, affected package
32+
- **Description**: Truncated to 500 characters
33+
- **Additional info**: CWEs, install paths, PURL, reference URLs
34+
- **Footer**: Scan timestamp, Grype version, DB version
35+
36+
## Inputs
37+
38+
| Input | Required | Default | Description |
39+
|-------|----------|---------|-------------|
40+
| `database-url` | Yes | - | PostgreSQL connection string (DSN) |
41+
| `data-repo-path` | Yes | `chef-vuln-scan-data` | Path to checked-out chef-vuln-scan-data repo |
42+
| `severities` | No | `Critical` | Comma-separated severity levels (Critical, High, Medium, Low) |
43+
| `teams-webhook-url` | No | - | Microsoft Teams incoming webhook URL |
44+
| `dry-run` | No | `false` | If `true`, print payloads without sending |
45+
46+
## Usage
47+
48+
### In a workflow
49+
```yaml
50+
- name: Checkout chef-vuln-scan-data
51+
uses: actions/checkout@v4
52+
with:
53+
repository: chef/chef-vuln-scan-data
54+
token: ${{ secrets.DATA_REPO_TOKEN }}
55+
path: chef-vuln-scan-data
56+
57+
- name: Notify new CVEs
58+
uses: ./.github/actions/notify-new-cves
59+
with:
60+
database-url: ${{ secrets.DATABASE_URL_RO }}
61+
data-repo-path: chef-vuln-scan-data
62+
severities: Critical,High
63+
teams-webhook-url: ${{ secrets.TEAMS_WEBHOOK_URL }}
64+
dry-run: false
65+
```
66+
67+
### Dry run for testing
68+
Set `dry-run: true` to print the notification payloads to the Actions log without sending them to Teams. Useful for:
69+
- Verifying query results
70+
- Testing card formatting
71+
- Validating enrichment logic
72+
73+
## Secrets Required
74+
75+
| Secret | Description | Scope |
76+
|--------|-------------|-------|
77+
| `DATABASE_URL_RO` | Postgres connection string (read-only user; only performs SELECT queries) | Org or repo |
78+
| `TEAMS_WEBHOOK_URL` | Teams incoming webhook URL (get from Teams channel connectors) | Org or repo |
79+
| `DATA_REPO_TOKEN` | PAT for checking out chef-vuln-scan-data (already exists) | Org or repo |
80+
81+
## Dispatcher Pattern
82+
83+
The notification logic is architected as a dispatcher so new channels can be added without changing the detection/enrichment logic:
84+
85+
```python
86+
def dispatch_notifications(notifications, teams_webhook=None, email_config=None, jira_config=None):
87+
if teams_webhook:
88+
send_teams_notification(notifications, teams_webhook)
89+
if email_config:
90+
send_email_notification(notifications, email_config) # Future
91+
if jira_config:
92+
create_jira_issues(notifications, jira_config) # Future
93+
```
94+
95+
To add a new channel, implement a new `send_*` function and add it to the dispatcher.
96+
97+
## Scheduled Workflow
98+
99+
The workflow that calls this action lives in [chef/chef-vuln-scan-orchestrator](https://github.com/chef/chef-vuln-scan-orchestrator):
100+
- Path: `.github/workflows/notify-new-cves.yml`
101+
- Schedule: 10:00 UTC daily (after nightly scans complete around 07:00 UTC)
102+
- Advantage: Uses existing `DATA_REPO_TOKEN` secret already configured in the orchestrator
103+
104+
## Verification Steps
105+
106+
### 1. Dry run
107+
Trigger `cd-notify-new-cves.yml` manually with `dry-run: true` — confirm formatted card payload appears in the Actions log.
108+
109+
### 2. Inject test row
110+
Manually insert a test CVE into `native_cve_details`:
111+
```sql
112+
INSERT INTO native_cve_details (
113+
scan_mode, product, channel, download_site,
114+
cve_id, severity, package_name, package_version,
115+
fix_available, fix_version,
116+
first_observed_at, last_seen_at
117+
) VALUES (
118+
'native', 'test-product', 'stable', 'commercial',
119+
'CVE-2025-99999', 'Critical', 'test-package', '1.0.0',
120+
false, NULL,
121+
NOW(), NOW()
122+
);
123+
```
124+
125+
Trigger dry run and confirm the test CVE is detected and enriched.
126+
127+
### 3. End-to-end
128+
Remove `dry-run: true` and trigger the workflow — confirm a Teams message arrives in the configured channel.
129+
130+
### 4. No false positives
131+
Re-run the workflow the next day when no new CVEs exist — confirm no messages are sent.
132+
133+
## Limitations
134+
135+
- **Scope**: Only native/modern products (habitat and container scans not yet supported)
136+
- **Deduplication**: A CVE appearing in multiple OS/arch platforms will send multiple notifications (one per unique product × CVE × package combination)
137+
- **Rate limiting**: Teams webhooks have rate limits (check Microsoft docs); may need throttling for large batches

‎.github/actions/notify-new-cves/SETUP.md‎

Lines changed: 255 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,255 @@
1+
# CVE Notifier Setup Guide
2+
3+
## Quick Start
4+
5+
The CVE notification system has been implemented in three components:
6+
7+
1. **notify.py** - Python script that queries the database, enriches with Grype data, and sends Teams notifications
8+
2. **action.yml** - GitHub Action wrapper around notify.py
9+
3. **cd-notify-new-cves.yml** - Scheduled workflow that runs daily at 10:00 UTC (in another repo).
10+
11+
## Files Created
12+
13+
```
14+
common-github-actions/
15+
├── .github/
16+
│ ├── actions/
17+
│ │ └── notify-new-cves/
18+
│ │ ├── action.yml # Action definition
19+
│ │ ├── notify.py # Core notification script
20+
│ │ └── README.md # Detailed documentation
21+
│ └── workflows/
22+
│ └── cd-notify-new-cves.yml # Daily scheduled workflow
23+
```
24+
25+
## Prerequisites
26+
27+
### 1. Create GitHub Secrets
28+
29+
Add these secrets to the `chef/chef-vuln-scan-orchestrator` repository (or at org level):
30+
31+
| Secret Name | Description | How to Get It |
32+
|------------|-------------|---------------|
33+
| `DATABASE_URL_RO` | Postgres connection string (read-only user) | From your RDS instance or infrastructure team |
34+
| `TEAMS_WEBHOOK_URL` | Teams incoming webhook URL | Create in Teams: Channel → Connectors → Incoming Webhook |
35+
| `DATA_REPO_TOKEN` | PAT for chef-vuln-scan-data | Should already exist (used by scan workflows) |
36+
37+
**Note**: The notification workflow only performs `SELECT` queries on `scan_runs`, `native_cve_details`, and `native_scan_results`. No write operations are performed—you can use a dedicated read-only database user for security.
38+
39+
#### Database URL Format
40+
41+
PostgreSQL connection string format:
42+
- Scheme: `postgresql`
43+
- Format: `<scheme>://<username>:<password>@<hostname>:<port>/<database>`
44+
- Default port: `5432`
45+
46+
Replace the angle-bracketed placeholders with your actual connection details.
47+
48+
#### Teams Webhook URL
49+
1. In Microsoft Teams, go to the channel where you want notifications
50+
2. Click the `...` menu → Connectors → Incoming Webhook
51+
3. Name: "Chef CVE Alerts" (or similar)
52+
4. Upload an icon (optional)
53+
5. Click "Create"
54+
6. Copy the webhook URL (starts with `https://progress.webhook.office.com/...`)
55+
7. Add as GitHub secret
56+
57+
### 2. Verify Database Schema
58+
59+
Ensure your database has the latest migrations applied:
60+
```bash
61+
cd chef-vuln-scan-db
62+
./scripts/migrate.sh
63+
```
64+
65+
Required tables:
66+
- `scan_runs` (stores run metadata)
67+
- `native_cve_details` (stores individual CVE findings)
68+
- `native_scan_results` (stores scan result aggregates)
69+
70+
### 3. Verify Data Repository Access
71+
72+
The workflow checks out `chef/chef-vuln-scan-data` using `DATA_REPO_TOKEN`. Verify this secret has read access.
73+
74+
## Testing
75+
76+
### Dry Run (Recommended First Step)
77+
78+
1. Go to the [chef-vuln-scan-orchestrator](https://github.com/chef/chef-vuln-scan-orchestrator) repo
79+
2. Navigate to Actions → Notify New CVEs
80+
3. Click "Run workflow"
81+
4. Set `dry-run` to `true`
82+
5. Set `severities` to `Critical` (or `Critical,High` for testing)
83+
6. Click "Run workflow"
84+
85+
This will:
86+
- Query the database
87+
- Enrich findings with Grype data
88+
- Print the Teams card payload to the Actions log
89+
- **NOT** send any actual notifications
90+
91+
Review the output to ensure:
92+
- CVEs are being detected correctly
93+
- Enrichment finds the Grype match data
94+
- Card formatting looks good
95+
96+
### Test with Injected CVE
97+
98+
If you want to test without waiting for a real CVE, inject a test row:
99+
100+
```sql
101+
-- Connect to your database
102+
psql $DATABASE_URL
103+
104+
-- Insert a test CVE
105+
INSERT INTO native_cve_details (
106+
scan_mode, product, channel, download_site,
107+
cve_id, severity, package_name, package_version,
108+
fix_available, fix_version,
109+
first_observed_at, last_seen_at
110+
) VALUES (
111+
'native', 'chef', 'stable', 'commercial',
112+
'CVE-2025-TEST', 'Critical', 'test-package', '1.0.0',
113+
false, NULL,
114+
NOW(), NOW()
115+
) ON CONFLICT DO NOTHING;
116+
117+
-- Verify it was inserted
118+
SELECT * FROM native_cve_details WHERE cve_id = 'CVE-2025-TEST';
119+
```
120+
121+
Then run a dry-run workflow. You should see the test CVE in the output.
122+
123+
**Important**: Clean up the test row after testing:
124+
```sql
125+
DELETE FROM native_cve_details WHERE cve_id = 'CVE-2025-TEST';
126+
```
127+
128+
### Live Test
129+
130+
Once dry-run looks good:
131+
132+
1. Run workflow again with `dry-run` set to `false`
133+
2. Check your Teams channel for the notification
134+
3. Verify the card displays correctly
135+
4. Click through the reference links to ensure they work
136+
137+
### Verify No False Positives
138+
139+
Run the workflow again immediately (or the next day when no new CVEs exist). It should:
140+
- Find 0 new CVEs
141+
- Print: "No new CVEs detected — no notifications to send"
142+
- **NOT** send any Teams messages
143+
144+
## Production Schedule
145+
146+
The workflow (located in `chef-vuln-scan-orchestrator/.github/workflows/notify-new-cves.yml`) can be enabled to run automatically daily at **10:00 UTC** by uncommenting the schedule:
147+
148+
```yaml
149+
schedule:
150+
- cron: '0 10 * * *'
151+
```
152+
153+
This is 3 hours after the nightly scans complete (~07:00 UTC), providing buffer time.
154+
155+
The schedule is commented out by default to allow testing with manual runs first.
156+
157+
## Customization
158+
159+
### Change Severity Threshold
160+
161+
Edit the workflow file to notify on High severity as well:
162+
```yaml
163+
severities: Critical,High
164+
```
165+
166+
Or run manually with custom severities via workflow_dispatch.
167+
168+
### Add Email Notifications
169+
170+
The dispatcher pattern makes this easy:
171+
172+
1. Add email configuration as a secret (JSON with SMTP settings)
173+
2. Implement `send_email_notification()` in notify.py
174+
3. Add email logic to `dispatch_notifications()`
175+
4. Add input to action.yml and workflow
176+
177+
### Add Jira Integration
178+
179+
Similar pattern:
180+
181+
1. Add Jira credentials as secrets
182+
2. Implement `create_jira_issues()` in notify.py
183+
3. Add to dispatcher
184+
4. Configure in action.yml/workflow
185+
186+
## Monitoring
187+
188+
### Check Workflow Runs
189+
190+
- Go to [chef-vuln-scan-orchestrator Actions](https://github.com/chef/chef-vuln-scan-orchestrator/actions)
191+
- Select "Notify New CVEs" workflow
192+
- Review recent runs for failures
193+
- Check logs for warnings about missing Grype matches
194+
195+
### Common Issues
196+
197+
**No CVEs found but expecting some:**
198+
- Check database: `SELECT * FROM native_cve_details WHERE first_observed_at >= NOW() - INTERVAL '25 hours'`
199+
- Verify scan workflows are running and inserting data
200+
- Check `last_seen_at` values are recent
201+
202+
**Grype match not found:**
203+
- Verify chef-vuln-scan-data repo has the grype.latest.json files
204+
- Check the file path pattern matches: `{scan_mode}/{product}/{channel}/{download_site}/**/scanners/grype.latest.json`
205+
- Look for warning in Actions log: "Could not find Grype match for..."
206+
207+
**Teams notification not arriving:**
208+
- Verify webhook URL is correct
209+
- Check Teams webhook hasn't been deleted/disabled
210+
- Look for HTTP error in Actions log
211+
- Test webhook manually with curl
212+
213+
**Database connection fails:**
214+
- Verify DATABASE_URL_RO secret is correct
215+
- Check database is accessible from GitHub Actions runners (security groups/firewalls)
216+
- Ensure the read-only user has SELECT permissions on the required tables
217+
218+
## Architecture
219+
220+
```
221+
┌─────────────────────────────────────────────────────────────┐
222+
│ GitHub Actions: cd-notify-new-cves.yml │
223+
│ Schedule: Daily at 10:00 UTC │
224+
└─────────────────────────────────────────────────────────────┘
225+
│
226+
▼
227+
┌─────────────────────────────────────────────────────────────┐
228+
│ Action: notify-new-cves │
229+
│ ┌─────────────────────────────────────────────────────┐ │
230+
│ │ notify.py │ │
231+
│ │ 1. Query Postgres for new CVEs (first_observed_at) │ │
232+
│ │ 2. Enrich with Grype match from chef-vuln-scan-data│ │
233+
│ │ 3. Format as Teams Adaptive Card │ │
234+
│ │ 4. Send to webhook (or dry-run) │ │
235+
│ └─────────────────────────────────────────────────────┘ │
236+
└─────────────────────────────────────────────────────────────┘
237+
│
238+
▼
239+
┌──────────────┐
240+
│ Microsoft │
241+
│ Teams │
242+
│ Channel │
243+
└──────────────┘
244+
```
245+
246+
## Next Steps
247+
248+
1. ✅ Create GitHub secrets (DATABASE_URL_RO, TEAMS_WEBHOOK_URL)
249+
2. ✅ Run dry-run test
250+
3. ✅ Inject test CVE and verify detection
251+
4. ✅ Run live test (send one notification to Teams)
252+
5. ✅ Monitor first scheduled run at 10:00 UTC tomorrow
253+
6. ⏳ Consider adding High severity notifications
254+
7. ⏳ Extend to Habitat scans (requires habitat_cve_details query)
255+
8. ⏳ Add email/Jira channels as needed

0 commit comments

Comments
 (0)