Skip to content

Commit dc023a2

Browse files
committed
fix(syslog): don't close the shared acquisition output channel
The syslog datasource's Stream() does `defer close(out)` on the pipeline output channel. That channel is shared: StartAcquisition() passes the same `out` to every configured datasource, and it is owned by the acquisition orchestrator, which never closes it (shutdown is driven by context cancellation, then drainChan). syslog is the only datasource that closes it. When syslog and another datasource run together, syslog's close races with the other datasource still sending. This is reproducible with the AppSec datasource: on shutdown/reload, an in-flight out-of-band request in appsec_runner's handleOutBandInterrupt does `r.outChan <- evt` after syslog has closed the channel, panicking with "send on closed channel" (seen in production on 1.7.8 under a syslog+appsec config). Remove the close. No consumer relies on it: runParse ranges via a `select { case <-ctx.Done(); case ev := <-input }` loop and drainChan is a non-blocking best-effort drain, so both terminate on context cancellation regardless. This aligns syslog with every other datasource (file, journalctl, docker, ...), none of which close the shared channel. The "never close out" contract is documented on the RestartableStreamer and Tailer interfaces, and a regression test asserts Stream() leaves the channel open after teardown.
1 parent 06be627 commit dc023a2

3 files changed

Lines changed: 54 additions & 2 deletions

File tree

pkg/acquisition/modules/syslog/run.go

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -43,8 +43,11 @@ func (s *Source) Stream(ctx context.Context, out chan pipeline.Event) error {
4343
})
4444

4545
g.Go(func() error {
46-
defer close(out)
47-
46+
// out is the shared pipeline output channel, fed by every configured
47+
// datasource; it is owned by the acquisition orchestrator, not by this
48+
// Stream(), so it must NOT be closed here. Doing so races with any other
49+
// datasource still sending on it (e.g. an in-flight appsec out-of-band
50+
// request) and panics with "send on closed channel".
4851
for {
4952
select {
5053
case <-ctx.Done():

pkg/acquisition/modules/syslog/syslog_test.go

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -167,3 +167,45 @@ disable_rfc_parser: true`,
167167
})
168168
}
169169
}
170+
171+
// TestStreamDoesNotCloseOutput is a regression test: Stream() must never close
172+
// the output channel. That channel is shared — the acquisition orchestrator
173+
// hands the same channel to every configured datasource and owns its lifecycle
174+
// (shutdown is driven by context cancellation, not by closing it). If syslog
175+
// closes it, any other datasource still sending on it panics with "send on
176+
// closed channel". We assert the channel stays open once Stream is torn down by
177+
// a context cancellation.
178+
func TestStreamDoesNotCloseOutput(t *testing.T) {
179+
ctx, cancel := context.WithCancel(t.Context())
180+
defer cancel()
181+
182+
s := Source{}
183+
config := `source: syslog
184+
listen_port: 4243
185+
listen_addr: 127.0.0.1`
186+
187+
err := s.Configure(ctx, []byte(config), log.WithField("type", ModuleName), metrics.AcquisitionMetricsLevelNone)
188+
require.NoError(t, err)
189+
190+
out := make(chan pipeline.Event)
191+
192+
go func() {
193+
_ = s.Stream(ctx, out)
194+
}()
195+
196+
// let the syslog server come up, then tear Stream down
197+
time.Sleep(500 * time.Millisecond)
198+
cancel()
199+
200+
// A closed channel yields (zero, false) on receive; an open, idle one hits the
201+
// default branch. Assert we never observe a close over the window. Receiving
202+
// (rather than sending) can't itself panic, so this is a safe probe.
203+
require.Never(t, func() bool {
204+
select {
205+
case _, ok := <-out:
206+
return !ok
207+
default:
208+
return false
209+
}
210+
}, time.Second, 50*time.Millisecond, "Stream() must not close the shared output channel")
211+
}

pkg/acquisition/types/types.go

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,10 @@ type Fetcher interface {
7676
// failures, but treat them as errors. The caller is responsible for supervising
7777
// Stream(), and restarting it as needed. There is currently no way to differentiate
7878
// retryable vs permanent errors.
79+
// - never close the output channel: it is shared and fed by every configured
80+
// datasource, so it is owned by the acquisition orchestrator, not by any single
81+
// Stream() implementation. Closing it here would race with (and can panic) any
82+
// other datasource still sending on it.
7983
type RestartableStreamer interface {
8084
// Start live acquisition (eg, tail a file)
8185
Stream(ctx context.Context, out chan pipeline.Event) error
@@ -84,6 +88,9 @@ type RestartableStreamer interface {
8488
// Tailer has the same pupose as RestartableStreamer (provide ongoing events) but
8589
// is responsible for spawning its own goroutines, and handling errors and retries.
8690
// New datasources are expected to implement RestartableStreamer instead.
91+
//
92+
// As with RestartableStreamer, the output channel is shared and orchestrator-owned:
93+
// an implementation must never close it.
8794
type Tailer interface {
8895
StreamingAcquisition(ctx context.Context, out chan pipeline.Event, acquisTomb *tomb.Tomb) error
8996
}

0 commit comments

Comments
 (0)