Furthermore to my previous post where I tried to compare events and commands, here I am presenting a scenario where they complement each other. As discussed, both of them are types of the messages and used for different purposes; while event messages are a backbone of the event-driven system, I believe that commands can add a lot of value. In this post, I will explore that further.
Let’s assume we have an Inventory system that listens to an event called ‘OrderProcessed’ from the Order system. When that event occurs, we want to adjust the inventory count.
Most of the service bus infrastructures promise to deliver a message at-least-once. That means the message broadcasting system wouldn’t care if the subscriber systems actually received the message.
As a subscriber, it is up to the inventory system to employ some resilience to ensure the message is processed. And if it fails more than the pre-determined allowed max retries, it will be moved to a DLQ (Dead Letter Queue). This is where our monitoring system kicks in and we will need some manual intervention. For something that needs to be processed again, we will be submitting a clone of the message. But that isn’t simple, there can be other listeners of this message, that may not be idempotent.
So what do we do? There are two techniques that can be used here.
Target A Subscriber:
In the clone of the message, add a flag (preferably in header) to specify the target audience i.e. all subscribers or a specific one. This requires all the listeners of the message to apply this filter. But it may or may not be possible depending on various factors like who else is listening or the technology they are using. If all the other systems are using the same tech-stack, you might be able to share some plumbing code for the subscribers at the cost of some level of coupling.
Another solution is to generate commands from your events. Where the event handler sends commands and gets done with it. Besides the usual transient failures or validations, the only other failure you need to consider is the command generation instead of complex business rules of your domain. Generally commands are meant to be used within a domain, so if you retry your command, you are not affecting any other domain except the current (where event is being handled).
This comes really handy if you are implementing the event sourcing with the hexagonal domain model. It requires a post of its own to explain how that works. I will try to write about it soon.
I hope you find these options helpful in solving some of the mysteries in your event driven systems.