Skip to content

Commit d05f3d7

Browse files
authored
Update index.md
1 parent 5d70f54 commit d05f3d7

1 file changed

Lines changed: 17 additions & 11 deletions

File tree

blog/batch-vs-stream-processing/index.md

Lines changed: 17 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -123,24 +123,30 @@ In batch, this doesn't exist as a problem. In streaming, it's a constant enginee
123123

124124

125125
## Hidden Cost #3 - Exactly-Once Is Harder Than It Sounds
126+
Handling failures in batch pipelines is usually predictable.
127+
If a batch job fails, you typically resolve the issue and rerun the pipeline from the beginning. Since the processing happens on bounded data, recovery is relatively straightforward.
126128

127-
In a batch pipeline, if a job fails, you rerun it. Simple.
129+
Streaming systems work very differently.
128130

129-
In a streaming pipeline, failure recovery is a distributed systems problem.
131+
In platforms like Kafka and Flink, data is continuously flowing through the system. If a streaming job crashes midway through processing, recovery becomes much more complex than simply restarting the job.
130132

131-
Say your Flink job crashes halfway through processing a Kafka topic partition. When it restarts, does it reprocess events it already processed? Does it skip events it hadn't gotten to yet? Does it process some events twice?
133+
For example, after recovery:
134+
- Should previously processed events be replayed?
135+
- Could some records get skipped unintentionally?
136+
- Is there a possibility that certain events are processed more than once?
132137

133-
This is the **exactly-once semantics** problem ensuring each event is processed exactly once, not zero times, not twice.
138+
This challenge is commonly addressed through **exactly-once processing guarantees**, where the goal is to ensure that every event affects the system exactly one time even during failures and restarts.
134139

135-
Getting this right requires:
136-
- Kafka consumer offset management
137-
- Flink checkpoint configuration
138-
- Idempotent writes to your output store
139-
- Careful handling of state during recovery
140+
Achieving reliable exactly-once behavior usually depends on several components working together correctly:
140141

141-
We had a production incident in our first month where a Flink job restart caused approximately 2,000 order events to be processed twice. Customers received duplicate status update notifications. Support tickets spiked. It took us two days to identify, patch, and backfill the correct state.
142+
- Proper Kafka offset management
143+
- Reliable Flink checkpointing and state recovery
144+
- Idempotent writes to downstream systems
145+
- Consistent state synchronization during failover scenarios
142146

143-
A batch job that fails leaves your data unchanged. You fix it, rerun, done. A streaming job that fails mid-stream can leave your data in a partially-updated state that's genuinely difficult to reason about.
147+
In practice, recovery bugs in streaming systems can have real operational impact. A single restart issue can lead to duplicate event processing, inconsistent downstream data, repeated customer notifications, or inaccurate analytics until the state is corrected.
148+
149+
Unlike batch systems, where failures often leave datasets untouched until rerun, streaming failures can leave systems in partially updated states that are significantly harder to debug and recover from.
144150

145151

146152
## Hidden Cost #4 - Testing Is a Different Discipline

0 commit comments

Comments
 (0)