AdvancedTopic 84 of 104Software & Computing
Message queues and streaming
Decouple services and process event streams.
In depth
Message queues (RabbitMQ, SQS) and log streams (Kafka) decouple producers and consumers, smooth bursts and enable asynchronous processing. Guarantees: at-most/at-least/exactly once; ordering per partition. Event streaming supports analytics and event sourcing.
Key points
- Pub/sub
- Consumer groups
- Backpressure
ExampleAn order service publishes events consumed by inventory and email services.
What is Message queues and streaming?
Decouple services and process event streams.
Why is it important to understand message queues and streaming?
Message queues decouple services so that one can send work without waiting for the other, which improves resilience and throughput.
What is the difference between a queue and a stream?
A queue usually removes a message once it is consumed, while a stream keeps a durable log that many consumers can read at their own pace.
Why is decoupling helpful?
A failure in one service does not immediately block the sender, and each side can scale and change on its own schedule.
How is Message queues and streaming used in practice?
Typical applications include sending order events to a queue so that the shipping service processes them later, buffering sensor readings during a burst so the database is not overwhelmed and processing a stream of events for analytics in real time.
What should I learn before Message queues and streaming?
Study "Caching and load balancing" first (topic 83 of 104 in Software & Computing). It covers the groundwork this topic builds on.
What are the key points of Message queues and streaming?
Pub/sub. Consumer groups. Backpressure.
Can you give an example of Message queues and streaming?
An order service publishes events consumed by inventory and email services.
What comes after Message queues and streaming?
The next topic is "Cryptography basics", which builds directly on this one.
How difficult is Message queues and streaming?
It is classed as Advanced in Software & Computing (topic 84 of 104). It is an advanced topic. It assumes you are comfortable with the intermediate topics in Software & Computing.