|
2 | 2 |
|
3 | 3 | ## Overview |
4 | 4 |
|
5 | | -This scenario demonstrates messaging with topics and queues using Amazon Simple Notification Service (Amazon SNS) and Amazon Simple Queue Service (Amazon SQS). The scenario shows how to create topics, queues, subscribe queues to topics, publish messages, and handle message filtering. |
| 5 | +Publish and subscribe is a mechanism for passing information. It’s used in social media, and it’s also used internally in software applications. A producer publishes a message, and the subscribers receive the message. In software, publish and subscribe notifications make message passing flexible and robust. The producers of messages are decoupled from the consumers of messages. |
6 | 6 |
|
7 | | -## What it demonstrates |
| 7 | +Use the sample code in this folder to explore publishing and subscribing to a topic by using filters and queues. This tutorial does not create a complete end-to-end application. Instead, you can use it to play around with a publish and subscribe architecture. |
8 | 8 |
|
9 | | -- Create SNS topics (standard and FIFO) |
10 | | -- Create SQS queues (standard and FIFO) |
11 | | -- Configure queue policies to allow SNS message delivery |
12 | | -- Subscribe queues to topics with optional message filtering |
13 | | -- Publish messages with attributes and FIFO-specific parameters |
14 | | -- Poll queues for messages and display results |
15 | | -- Clean up resources (delete queues, unsubscribe, delete topics) |
| 9 | +You can create an Amazon SNS topic and subscribe two Amazon SQS queues to the topic. You can enable FIFO (First-In-First-Out) queueing, and you can add filtered subscriptions. Then, you can publish messages to the topic and see the results in the queues. |
16 | 10 |
|
17 | | -## Files |
| 11 | +You can publish and subscribe using Amazon SNS alone. But combining Amazon SNS with Amazon SQS gives you more flexibility in how the messages are consumed. |
18 | 12 |
|
19 | | -- `topics_and_queues_scenario.py` - Main scenario orchestration |
20 | | -- `sns_wrapper.py` - SNS operations wrapper class |
21 | | -- `sqs_wrapper.py` - SQS operations wrapper class |
22 | | -- `requirements.txt` - Python dependencies |
23 | | -- `test/` - Integration tests |
| 13 | +Amazon SNS is a push service. It pushes to endpoints such as email addresses, mobile application endpoints, or SQS queues. (For a full list of endpoints, see [SNS event destinations](https://docs.aws.amazon.com/sns/latest/dg/sns-event-destinations.html)). |
24 | 14 |
|
25 | | -## Prerequisites |
| 15 | +With Amazon SQS, messages are received from a queue by polling. With polling, the subscriber receives messages by calling a receive message API. Any code can poll the queue. Also, the messages stay in the queue until you delete them. This gives you more flexibility in how the messages are processed. |
26 | 16 |
|
27 | | -- Python 3.8 or later |
28 | | -- AWS credentials configured (via AWS CLI, environment variables, or IAM roles) |
29 | | -- Appropriate AWS permissions for SNS and SQS operations |
| 17 | +The sample code builds a command line application that asks you for input. This is implemented in multiple programming languages, and the interface can vary slightly between languages. The following shows the interface for the Python implementation. |
30 | 18 |
|
31 | | -## Setup |
| 19 | +### Create an SNS topic |
32 | 20 |
|
33 | | -1. Install dependencies: |
34 | | -```bash |
35 | | -pip install -r requirements.txt |
| 21 | +``` |
| 22 | +Would you like to work with FIFO topics? (y/n) |
| 23 | +``` |
| 24 | + |
| 25 | +You configure FIFO (First-In-First-Out) topics when you create them. Choosing a FIFO topic enables other options, too. To learn more, see [FIFO topics example use case](https://docs.aws.amazon.com/sns/latest/dg/fifo-example-use-case.html). |
| 26 | + |
| 27 | + |
| 28 | +``` |
| 29 | +Use content-based deduplication instead of a deduplication ID? (y/n) |
36 | 30 | ``` |
37 | 31 |
|
38 | | -2. Ensure AWS credentials are configured: |
39 | | -```bash |
40 | | -aws configure |
| 32 | +Deduplication is only available for FIFO topics. Deduplication prevents the subscriber from responding more than once to events that are determined to be duplicates. If a message gets published to an SNS FIFO topic and it’s found to be a duplicate within the five-minute deduplication interval, the message is accepted but not delivered. For more information, see [Message deduplication for FIFO topics](https://docs.aws.amazon.com/sns/latest/dg/fifo-message-dedup.html). |
| 33 | + |
| 34 | +Content-based deduplication uses a hash of the content as a deduplication ID. If content-based deduplication is not enabled, you must include a deduplication ID with each message. |
| 35 | + |
| 36 | +``` |
| 37 | +Enter a name for your SNS topic: |
41 | 38 | ``` |
42 | 39 |
|
43 | | -## Running the scenario |
| 40 | +Topic names can have 1-256 characters. They can contain uppercase and lowercase ASCII letters, numbers, underscores, and hyphens. If you chose a FIFO topic, the application automatically adds a “.fifo” suffix, which is required for FIFO topics. |
44 | 41 |
|
45 | | -```bash |
46 | | -python topics_and_queues_scenario.py |
| 42 | +### Create two SQS queues |
| 43 | + |
| 44 | +Now, configure two SQS queues to subscribe to your topic. Separate queues for each subscriber can be helpful. For |
| 45 | +instance, you can customize how messages are consumed and how messages are filtered. |
| 46 | + |
| 47 | +``` |
| 48 | +Enter a name for an SQS queue. |
47 | 49 | ``` |
48 | 50 |
|
49 | | -## Scenario workflow |
| 51 | +Queue names can have 1-80 characters. They can contain uppercase and lowercase ASCII letters, numbers, underscores, and hyphens. If you chose a FIFO topic, the application automatically adds a “.fifo” suffix, which is required for FIFO queues. |
50 | 52 |
|
51 | | -### 1. Topic Setup |
52 | | -- Choose between standard or FIFO topic |
53 | | -- For FIFO topics, configure deduplication options |
54 | | -- Create the SNS topic |
55 | 53 |
|
56 | | -### 2. Queue Setup |
57 | | -- Create SQS queues (matching topic type) |
58 | | -- Configure queue policies to allow SNS message delivery |
59 | | -- Subscribe queues to the topic with optional message filtering |
| 54 | +``` |
| 55 | +Filter messages for "<queue name>.fifo"s subscription to |
| 56 | +the topic "<topic name>.fifo"? (y/n) |
| 57 | +``` |
| 58 | + |
| 59 | +If you chose FIFO topics, you can add a filter to the queue’s topic subscription. There are many ways to filter a topic. In this example code, you have the option to filter by a predetermined selection of attributes. For more information about filters, see [Message filtering for FIFO topics](https://docs.aws.amazon.com/sns/latest/dg/fifo-message-filtering.html). |
60 | 60 |
|
61 | | -### 3. Message Publishing |
62 | | -- Publish messages to the topic |
63 | | -- For FIFO topics, specify message group ID and optional deduplication ID |
64 | | -- Add tone attributes for message filtering |
65 | 61 |
|
66 | | -### 4. Message Polling |
67 | | -- Poll each queue for messages |
68 | | -- Display message contents |
69 | | -- Delete messages after processing |
| 62 | +``` |
| 63 | +You can filter messages by one or more of the following "tone" attributes. |
| 64 | +1. cheerful |
| 65 | +2. funny |
| 66 | +3. serious |
| 67 | +4. sincere |
| 68 | +Enter a number (or enter zero to stop adding more) |
| 69 | +``` |
| 70 | + |
| 71 | +If you add a filter, you can select one or more “tone” attributes to filter by. When you’re done, enter “0’” to continue. |
70 | 72 |
|
71 | | -### 5. Cleanup |
72 | | -- Option to delete queues |
73 | | -- Unsubscribe from topics |
74 | | -- Option to delete topics |
| 73 | +The application now prompts you to add the second queue. Repeat the previous steps for the second queue. |
75 | 74 |
|
76 | | -## FIFO Features |
| 75 | +The following diagram shows the topic and queue options. |
| 76 | + |
77 | 77 |
|
78 | | -When using FIFO topics and queues, the scenario demonstrates: |
| 78 | +After you create the topic and subscribe both queues, the application lets you publish messages to the topic. |
79 | 79 |
|
80 | | -- **Message Ordering**: Messages within the same message group are delivered in order |
81 | | -- **Deduplication**: Prevents duplicate message delivery using deduplication IDs or content-based deduplication |
82 | | -- **Message Filtering**: Filter messages by tone attribute (cheerful, funny, serious, sincere) |
83 | 80 |
|
84 | | -## Error Handling |
| 81 | +``` |
| 82 | +Enter a message text to publish. |
| 83 | +``` |
85 | 84 |
|
86 | | -The scenario includes comprehensive error handling: |
87 | | -- AWS service errors are caught and logged |
88 | | -- User-friendly error messages |
89 | | -- Graceful cleanup on failures |
90 | | -- Validation of user inputs |
| 85 | +All configurations include a message text. |
91 | 86 |
|
92 | | -## Architecture |
93 | 87 |
|
94 | 88 | ``` |
95 | | -┌─────────────────┐ ┌─────────────────┐ |
96 | | -│ SNS Topic │ │ SQS Queue 1 │ |
97 | | -│ ├────┤ │ |
98 | | -│ (Standard or │ │ (with optional │ |
99 | | -│ FIFO) │ │ filtering) │ |
100 | | -└─────────────────┘ └─────────────────┘ |
101 | | - │ |
102 | | - │ ┌─────────────────┐ |
103 | | - └──────────────┤ SQS Queue 2 │ |
104 | | - │ │ |
105 | | - │ (with optional │ |
106 | | - │ filtering) │ |
107 | | - └─────────────────┘ |
| 89 | +Enter a message group ID for this message. |
108 | 90 | ``` |
109 | 91 |
|
110 | | -## Testing |
| 92 | +If this is a FIFO topic, then you must include a group ID. The group ID can contain up to 128 alphanumeric characters `(a-z, A-Z, 0-9)` and punctuation `(!"#$%&'()*+,-./:;<=>?@[\]^_``{|}~)`. |
| 93 | +For more information about group IDs, see [Message grouping for FIFO topics](https://docs.aws.amazon.com/sns/latest/dg/fifo-message-grouping.html). |
| 94 | + |
111 | 95 |
|
112 | | -Run the integration tests: |
113 | | -```bash |
114 | | -cd test |
115 | | -python -m pytest test_topics_and_queues_scenario.py -v |
116 | 96 | ``` |
| 97 | +Enter a deduplication ID for this message. |
| 98 | +``` |
| 99 | + |
| 100 | +If this is a FIFO topic and content-based deduplication is not enabled, then you must enter a deduplication ID. The message deduplication ID can contain up to 128 alphanumeric characters `(a-z, A-Z, 0-9)` and punctuation `(!"#$%&'()*+,-./:;<=>?@[\]^_``{|}~)`. |
| 101 | + |
| 102 | + |
| 103 | +``` |
| 104 | +Add an attribute to this message? (y/n) y |
| 105 | +``` |
| 106 | + |
| 107 | +If you added a filter to one of the subscriptions, you can choose to add a filtering attribute to the message. |
| 108 | + |
| 109 | + |
| 110 | +``` |
| 111 | +1. cheerful |
| 112 | +2. funny |
| 113 | +3. serious |
| 114 | +4. sincere |
| 115 | +Enter a number for an attribute: |
| 116 | +``` |
| 117 | + |
| 118 | +Select a number for an attribute. |
| 119 | + |
| 120 | + |
| 121 | +``` |
| 122 | +Post another message? (y/n) |
| 123 | +``` |
| 124 | + |
| 125 | +You can post as many messages as you want. |
| 126 | + |
| 127 | +When you are done posting messages, the application polls the queues and displays their messages. |
| 128 | + |
| 129 | + |
| 130 | +## ⚠️ Important |
| 131 | + |
| 132 | +* Running this code might result in charges to your AWS account. |
| 133 | +* Running the tests might result in charges to your AWS account. |
| 134 | +* We recommend that you grant your code least privilege. At most, grant only the minimum permissions required to perform the task. For more information, see [Grant least privilege](https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html#grant-least-privilege). |
| 135 | +* This code is not tested in every AWS Region. For more information, see [AWS Regional Services](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services). |
| 136 | + |
| 137 | +## Run the examples |
| 138 | + |
| 139 | +### Prerequisites |
| 140 | + |
| 141 | +Before using the code examples, first complete the installation and setup steps of [Getting started](https://docs.aws.amazon.com/sdk-for-cpp/v1/developer-guide/getting-started.html) in the AWS SDK for |
| 142 | +C++ Developer Guide. |
| 143 | + |
| 144 | + |
| 145 | +### Instructions |
117 | 146 |
|
118 | | -## Clean up |
119 | 147 |
|
120 | | -The scenario provides interactive cleanup options at the end. You can also manually clean up resources: |
| 148 | +Running this example requires AWS Identity and Access Management (IAM) permissions for both SNS and SQS. |
121 | 149 |
|
122 | | -1. Delete SQS queues from the AWS Console |
123 | | -2. Delete SNS topics from the AWS Console |
124 | | -3. Subscriptions are automatically deleted when queues are deleted |
| 150 | +## Additional resources |
125 | 151 |
|
126 | | -## Related AWS Services |
| 152 | +* [Amazon SNS Developer Guide](https://docs.aws.amazon.com/sns/latest/dg/welcome.html) |
| 153 | +* [Amazon SNS API Reference](https://docs.aws.amazon.com/sns/latest/api/welcome.html) |
| 154 | +* [Amazon SQS Developer Guide](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/welcome.html) |
| 155 | +* [Amazon SQS API Reference](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/Welcome.html) |
127 | 156 |
|
128 | | -- [Amazon SNS Documentation](https://docs.aws.amazon.com/sns/) |
129 | | -- [Amazon SQS Documentation](https://docs.aws.amazon.com/sqs/) |
130 | | -- [SNS Message Filtering](https://docs.aws.amazon.com/sns/latest/dg/sns-message-filtering.html) |
131 | | -- [FIFO Topics](https://docs.aws.amazon.com/sns/latest/dg/sns-fifo-topics.html) |
|
0 commit comments