Commit 9ee6410
authored
[CWS] Fix panic on closed channel during APIServer shutdown (#47047)
### What does this PR do?
Wait for the `APIServer.start()` goroutine to finish before stopping pipeline providers, preventing a race where events are sent on a closed `logChan`.
### Motivation
During system-probe shutdown, `CWSConsumer.Stop()` calls `c.cancelFnc()` to cancel the Go context, and then `c.apiServer.Stop()`. `c.apiServer.Stop()` then calls `a.stopper.Stop()` to close the `logChan` channel. However, the `APIServer.start()` goroutine was not tracked by any synchronization primitive, so after the Go context was cancelled the goroutine could still be in the middle of a dequeue cycle while the `logChan` channel was already closed.
This resulted in the following panic:
```
panic: send on closed channel
goroutine 781 [running]:
github.com/DataDog/datadog-agent/pkg/security/reporter.(*RuntimeReporter).ReportRaw(0x4000e84e20, {0x4003c26000, 0x3d43, 0x4000}, {0x0, 0x0}, {0x40009f7b48?, 0x1a15a98?, 0x0?}, {0x400209a880, ...})
github.com/DataDog/datadog-agent/pkg/security/reporter/reporter.go:39 +0x1d0
github.com/DataDog/datadog-agent/pkg/security/module.(*DirectEventMsgSender).Send(0x400278a2a0, 0x400295d7c0, 0x3a?)
github.com/DataDog/datadog-agent/pkg/security/module/msg_sender.go:100 +0x138
github.com/DataDog/datadog-agent/pkg/security/module.(*APIServer).start.func1(0x40016c3e00, 0x1)
github.com/DataDog/datadog-agent/pkg/security/module/server.go:386 +0x400
github.com/DataDog/datadog-agent/pkg/security/module.(*APIServer).dequeue.func1(...)
github.com/DataDog/datadog-agent/pkg/security/module/server.go:256
github.com/DataDog/datadog-agent/pkg/security/module.slicesDeleteUntilFalse(...)
github.com/DataDog/datadog-agent/pkg/security/module/server.go:280
github.com/DataDog/datadog-agent/pkg/security/module.(*APIServer).dequeue(0x40019afa20, {0x4003e2bf20?, 0x48e7fd6f8e4?, 0x401af60?}, 0x40009f7f30)
github.com/DataDog/datadog-agent/pkg/security/module/server.go:248 +0x1d8
github.com/DataDog/datadog-agent/pkg/security/module.(*APIServer).start(0x40019afa20, {0x2ba4608, 0x400225d0e0})
github.com/DataDog/datadog-agent/pkg/security/module/server.go:337 +0x80
created by github.com/DataDog/datadog-agent/pkg/security/module.(*APIServer).Start in goroutine 1
github.com/DataDog/datadog-agent/pkg/security/module/server.go:412 +0x1d0
```
### Describe how you validated your changes
### Additional Notes
Co-authored-by: yoann.ghigoff <yoann.ghigoff@datadoghq.com>1 parent 3c1e0ca commit 9ee6410
1 file changed
Lines changed: 11 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
176 | 176 | | |
177 | 177 | | |
178 | 178 | | |
| 179 | + | |
179 | 180 | | |
180 | 181 | | |
181 | 182 | | |
| |||
409 | 410 | | |
410 | 411 | | |
411 | 412 | | |
412 | | - | |
| 413 | + | |
| 414 | + | |
| 415 | + | |
| 416 | + | |
| 417 | + | |
413 | 418 | | |
414 | 419 | | |
415 | 420 | | |
| |||
685 | 690 | | |
686 | 691 | | |
687 | 692 | | |
688 | | - | |
| 693 | + | |
| 694 | + | |
689 | 695 | | |
| 696 | + | |
| 697 | + | |
| 698 | + | |
690 | 699 | | |
691 | 700 | | |
692 | 701 | | |
| |||
0 commit comments