Skip to content

Allow opting out of content-based FIFO deduplication for raw sends#1038

Draft
mensfeld wants to merge 1 commit into
mainfrom
fix/fifo-dedup-opt-out
Draft

Allow opting out of content-based FIFO deduplication for raw sends#1038
mensfeld wants to merge 1 commit into
mainfrom
fix/fifo-dedup-opt-out

Conversation

@mensfeld

Copy link
Copy Markdown
Collaborator

Queue#add_fifo_attributes! always set message_deduplication_id to a SHA256 of the body when none was provided, so two raw sends of an identical body within SQS's 5-minute window (Worker.perform_async, Queue#send_message/#send_messages) were silently deduplicated - the second dropped. ActiveJob got an opt-out in #1017
(active_job_fifo_message_deduplication) but the raw path had none.

Add the symmetric Shoryuken.fifo_message_deduplication flag (default true, so behaviour is unchanged). When set false, Shoryuken no longer auto-generates the dedup id; callers provide message_deduplication_id themselves or enable ContentBasedDeduplication on the queue.

Queue#add_fifo_attributes! always set message_deduplication_id to a
SHA256 of the body when none was provided, so two raw sends of an
identical body within SQS's 5-minute window (Worker.perform_async,
Queue#send_message/#send_messages) were silently deduplicated - the
second dropped. ActiveJob got an opt-out in #1017
(active_job_fifo_message_deduplication) but the raw path had none.

Add the symmetric Shoryuken.fifo_message_deduplication flag (default
true, so behaviour is unchanged). When set false, Shoryuken no longer
auto-generates the dedup id; callers provide message_deduplication_id
themselves or enable ContentBasedDeduplication on the queue.
@coderabbitai

coderabbitai Bot commented Jun 24, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: f163598b-7f89-4710-b39d-2de27e1cb403

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/fifo-dedup-opt-out

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant