skip to content
anntz.com

messaging 101

backend, system-design, message-queues

messaging protocols enable reliable, scalable, and asynchronous communication between components of a distributed system.

Table of Contents

messaging 101

Messaging is a set of protocol and technologies that enables communication between different components of a distributed system. It is crucial for allowing various parts of an application or different applications to communicate with each other in a reliable, scalable, and asynchronous manner.

the core concepts

  1. message: A message is a data packet that contains information to be shared between different parts of a system. It typically consists of a payload (the actual data) and a header (metadata about the data).

  2. producer (/publisher) and consumer (subscriber): In messaging systems, a producer sends messages, and a consumer receives them. In publish-subscribe models, producers are called publishers, and consumers are called subscribers.

  3. queue: This is a temporary storage that holds messages until they can be processed by a consumer. It helps in managing load and ensuring message delivery even if the consumer is busy or unavailable.

  4. message broker: A software component that mediates communication between producers and consumers. It ensures that messages are delivered from producers to the appropriate consumers.

messaging models

  1. point-to-point (/queue): Messages are placed in a queue and consumed by a single consumer. It’s like sending an email; once a message is read, it’s not available to other consumers.

  2. publish-subscribe (/pub-sub): Messages are published to a specific topic. Multiple consumers can subscribe to this topic and receive messages. It’s akin to a newsletter where all subscribers receive the same message.

protocols and standards

  • AMQP (Advanced Message Queuing Protocol): A protocol for message-oriented middleware that ensures interoperability between different systems.
  • MQTT (Message Queuing Telemetry Transport): A lightweight messaging protocol, ideal for IoT devices with limited resources.
  • JMS (Java Message Service): A Java-based messaging standard that allows application components to create, send, receive, and read messages.

applications and use cases

  1. Decoupling of Systems: Messaging allows different systems to operate independently without direct knowledge of each other, enhancing system scalability and maintainability.

  2. Load Balancing: By distributing messages across multiple consumers, messaging technology can balance the load and improve the overall performance of the system.

  3. Asynchronous Communication: It allows parts of a system to communicate without waiting for responses, improving efficiency and responsiveness.

  4. Fault Tolerance: Messaging systems can store messages in queues, ensuring that no data is lost in case of system failures.

  • Apache Kafka: A distributed streaming platform often used for building real-time data pipelines and streaming applications.
  • RabbitMQ: A popular open-source message broker that supports multiple messaging protocols. Extras: differences between Apache Kafka and RabbitMQ
  • Amazon SQS (Simple Queue Service): A managed message queuing service offered by AWS.