Should you go after ActiveMQ, RabbitMQ or similar? What about Apache Kafka, Amazon Kinesis, etc.…? Your choice, as all of them, will have consequences so you want to be as sure as possible before committing. Before drilling down let us simplify agreeing that there are two groups: “MQ” which include products like IBM MQ or ActiveMQ, and there’s Kafka and the products alike.
First let us get out of the way that both groups of products have proven record of be able to provide scalability and high availability. There are not short either of security or management capabilities. At the core both products let us implement architectures that are loosely coupled and provide asynchronous support by having components communicate via messages.
Now, what are the messages meant to represent? Two main options here:
- A command, query, or operation. In this case component A wants to have somebody else do something so it knows it needs to post a message in a specific place. In many cases, there will be a reply.
- A notification representing that something has occurred. An event and the associated event data. In this option, component A did something in post a message in a specific to shout “I did this.”
If your system inequivalent needs to do option #1, you can stop here and assume MQ platforms are the right choice. But if it is not as crystal clear, let us review how are messages going to be processed:
- Message needs to be read only once and removed from the queue: In this case, we are looking at MQ with a destructive consumption that warranties that only one consume will read a message and that message will only be processed once.
- Messages needs to be read once by multiple consumers: In this case your best set with a Kafka-like product. Messages can be read once by each consumer as they remain available in the topic until conditions defined on the producer side. This is the core advantage the Kafka brings, as it let us plug consumers as needed without impacting what others.
You may find yourself needed to address a combination of these options. In that case you need to investigate the specific MQ system that you may use. MQ systems also have support for Topics, so you should do be able to communicate events.