By Uday

When working in cloud-based infrastructure, We need to know about the queuing services. Similarly, there is one queuing service on Amazon that is called SQS (Simple Queue Service)

Hey Readers, All right let’s start with Amazon SQS, How does it work in the background?

How does Amazon SQS works?

Amazon Simple Queue Service (SQS) is a message queuing service that enables you to manage asynchronous communication between applications or microservices.

If you are new to queuing services, imagine this process as a communication between two persons through WhatsApp or a normal SMS network. There always should be at least two participants, the producer (*the one who sends the messages*) and the consumer(the one who receives it). For direct communication (mobile calls) both the participants must be online and available.
There are three main component’s producer, intermediary and consumer components.

Asynchronous communication is similar to messaging through a platform or even SMS. In this case, the producer sends a message and if the consumer is not available at the moment, the message will wait on an intermediary component, As soon as the consumer is available, the message pulling will start, and it processes the message and delete it to reduce the duplicated pulling.

In the Amazon SQS message can be processed in two ways, I.e., Standard Queues, FIFO Queues

1. Standard Queue: Amazon SQS offers a standard queue as a default queue type. This queue supports a nearly unlimited number of API calls per second, SQS stores copies of your messages on multiple servers for redundancy and high availability. When you receive or delete a message, one of the servers that stores a copy of the message may be unavailable on rare occasions. If this occurs, the message copy is not deleted from the unavailable server, and you may receive that message copy again when you receive messages.
   Make your applications idempotent (they should not be affected adversely when processing the same message more than once).

In these standard queues, there is no message order management, the messages can process in shuffled order.

2. FIFO Queue: FIFO (first-in-first-out) model queues are used to process messages only once and in a strict order, Example of the situation when can we use this FIFO queue

When you need to send a message to your friends through WhatsApp then we need to use FIFO queues for managing the messages in orders, If wont maintian the order of messgaes then there is a chance for miss communcations. So avoiding of this miss communication and strict process we have this FIFO queues

FIFO queues also provide exactly-once processing but have a limited number of transactions per second (TPS).

On the creation of the FIFO queue, We need to remember that the queue name must end with the .fifo

All right readers, Now we have a bit of clarity on queues. Here are several important terms to understand before you start working with Amazon SQS

1. SQS Visibility Timeout: This is a short period of time when a message is protected from receiving process by other consumers, you can configure it from 0 to 12 hours. For more information, read these docs

2. Dead-Letter Queues: It’s useful for messages that can’t be processed successfully for some reason. Failed messages will move to a dead-letter queue automatically, you just need to specify the conditions. Connect the source queue with a dead-letter one and set the maximum received count, the number of times a message was unable to be processed and
   when the limit is reached, the message is redirected to the dead-letter queue and is removed from the source queue. You can also configure notifications for failed messages to instantly check those in dead-letter queues and analyze what went wrong.
   Dead-letter queues are a good place to start when investigating the contents of failed messages and the logs of exceptions that caused them to fail.

3. Message retention period: The amount of time that Amazon SQS keeps messages in the queue. Messages are kept in the queue for four days by default.
You can set a queue to keep messages for up to 14 days.

4. Delivery delay: The amount of time Amazon SQS will wait before delivering a message that has been queued.

5. Maximum message size: The maximum message size for this queue. By default, it should be 256 KB

6. SQS short and long polling: These are the two types of messages that are consumed. When the waiting time for the ReceiveMessage API action exceeds zero, long polling occurs.
If the waiting time equals zero, the polling is short.
By default, short polling is used.
It returns a faster response because it asks a subset of Amazon SQS servers (where the queue stores and processes messages), so you may need to issue multiple subsequent requests to get all messages from all servers.
Amazon SQS uses long polling to wait for messages (until they time out) and then asks all servers.
A request takes longer to process in this case, but it is more likely that it will not be empty.

SQS Use Cases

1. Work Queues:
   Decouple components of a distributed application that may not all be performing the same amount of work at the same time.

2. Buffer and Batch Operations: Increase the architecture’s scalability and reliability, and smooth out temporary volume spikes without losing messages or increasing latency.

Queue and Message Identifiers

Queue URLs: 1. Within the same AWS account, each queue is identified by a unique queue name.
2. SQS assigns a Queue URL identifier to each queue, such as http://sqs.us-east-1.amazonaws.com/123456789012/mymainqueue.
3. Any operation on the Queue requires the Queue URL.

Message ID: 1. Message IDs can be used to identify messages.
2. Each message is assigned a message ID by the system, which SQS returns to with the SendMessage response.
3. To delete a message, use the message’s receipt handle rather than the message ID.
4. The maximum length for a message ID is 100 characters.

Body: The queue message should be in-said the body, it should be in the form of JSON text.

Receipt Handle: 1. Instead of the message itself, a receipt handle connected to the act of receiving the message is returned when a message is retrieved from a queue.
2. To delete a message or change the visibility of a message, you need the receipt handle rather than the message ID.
3. If a message is received more than once, a different receipt handle is assigned for each receipt, and the most recent receipt handle should always be used.

All right now we all know about the SQS service, Let’s jump into the creation of SQS through the AWS console

Pre-requirements, We need one active AWS account

Creating Queue through AWS Console

1. Login into that AWS Account which will lead into the services console, and select the specific region in the top right corner, later search for the SQS service in the search section and select the SQS Service it will redirect to the create queue console.

Search for the SQS Service

2. In this create queue console there is a Details section, it will talk about which type of queue we need and what is that queue name you need to call it.

You can’t change the queue type after you create a queue.

Let’s select the queue type and give the meaningful name for the main queue

Queue type and Name

3. The configuration section is pre-filled with default values that are suitable for basic queue operation.

Configure the following values to customize your queue:
3a. Visibility timeout: the length of time that a message received from a queue (by one consumer) will not be visible to the other message consumers.
3b. Message retention period: the amount of time that Amazon SQS retains a message that does not get deleted.
3c. Delivery delay: the amount of time to delay the first delivery of each message added to this queue.
3d. Maximum message size: the maximum message size for this queue.
3e. Receive message wait time: the maximum amount of time that polling will wait for messages to become available.
3f. Content-based deduplication: Amazon SQS can automatically create deduplication IDs, based on the body of the message.

For now, we can keep all these attributes values as default values

Configuration settings

4. Access Policy, The access policy defines the accounts, users and roles that can access this queue, and the actions that are allowed. You can configure basic and advanced settings. We can select a basic setting and move on to the next session.

Access Policy

5. Redrive allow policy, The redrive allow policy defines which source queues can use this queue as the dead-letter queue. Select disable

6. Encryption, When you enable server-side encryption (SSE), Amazon SQS encrypts all messages that are sent to this queue. You can select disable option

7. Dead-Letter Queue: If a message can’t be consumed successfully, you can send it to a dead-letter queue (DLQ). Dead-letter queues let you isolate problematic messages to determine why they are failing. Now we don’t want any dead letter queue, so we can go with disable option.

That’s all hit the create queue option, it will create one queue and redirect it to the main queue console, where you can see that all queue configurations and details

How to send messages to queue:

After completion of the queue creation go to the queue and select the queue, it will show the send and receive message option in the top right corner as shown in the below image.

It will redirect you to the text editor, write the text that you need to send in that message and hit send message option so that all your message is sent successfully.

All right, that’s all about this blog, let’s catch up with one more new blog keep supporting and following me for more additional updates about the AWS and Python Blogs, On Twitter and LinkedIn

Conclusion: Message queues allow these distributed applications to communicate and coordinate.
Message queues can greatly simplify decoupled application coding while also improving performance, reliability, and scalability.

Thank you for devoting some of your free time to read this blog post. Have a wonderful day :)