Events on the Outside vs Events on the Inside

Back in 2005, Pet Helland has written this excellent white paper called “Data on the Outside versus Data on the Inside“. It explores the concept of time “Then and Now” on the data. One of the key take-aways is the application of the theory of the real world on the data.

By the time you see a distant object, it may have changed!

By the time you see a message, the data may have changed!

Pet sees the data in two states i.e. Then and Now. Just by being aware of the state of the data, one can make an informed decision while producing/consuming it. Mainly it sees the data within a system (bounded context) as a state of “now”, where data is in a highly consistent state. However, when the same data is outside the system, either shared as a response to a request or a command from an external system, it is in a state of “then”.

  • The response of a request carries data as of the time response was generated from the producing system.
  • A command from the external system requests a future state of the data within the consuming system.
Data on the Outside vs Data on the Inside

Now State – Data On the Inside

This data is within a single bounded context, which gives it unique features and constraints as discussed in the next points.

  • Transaction scope: Data being in a single bounded context, enables the read and write operations to be performed within a transaction scope.
  • Consistent and Mutable: All invariants checks are made on the persisted state within a scope of a transaction scope, this means we can work on data considering it a current state and keep the state always consistent. This also allow the data to be mutable as well.

Then State – Data On the Outside

Data, when shared outside the bounded context, is considered a “then” state as it is shared outside the bounded context as well as transaction boundary. Incoming commands request a future state so they are also in a “then” state.

  • Immutable / Read-Only: Once shared outside the bounded context, the data within the bounded context can change but the copy that is already shared outside is read-only.
  • Grows stale with time: The other side effect to be a disconnected copy of the source data, it starts growing stale with time.

The above two characteristics of the “then” state data make it a good candidate of uniquely version each copy e.g. It is more accurate to say “Helard Sun of Sunday, 7th March 2021” instead of just “Herald Sun”

Applying state on Events

When implementing an event-driven system, we have to put careful thoughts into what events will be produced. There are few popular ways to identify events that will be produced:

Applying the learnings from Pet’s paper, we can categorize these events into Inside vs Outside events, sometimes referred to as Domain Events and Integration Events. Domain Events are internal to a bounded context and when generated, they live in the “Now” state of the domain until they are persisted, hence more like “Data on the Inside”. On the other hand, integration events are published outside of the bounded context, mostly for the consumption of the external systems. They start growing stale from the moment they are published to the outer world, which places them in the “Data on the Outside” camp.

Domain Events – Events on The Inside:

  • Domain events are within a bounded context for inter aggregates communication
  • In-memory or persisted (event sourcing)
  • Consumer, producer often in the same transaction scope.
  • Part of mutable state but they are immutable themselves
  • Can be lightweight: refer entities Ids within a bounded context
  • Subject to faster evolution
  • Event Sourcing Example
  • In Process Events

Integration Events – Events on the Outside

  • Inter bounded context communication
  • Immutable, cacheable, read-only
  • Event body is formal contract: Unique ID (aggregate id + version)
  • Derived off domain events or the domain code as a side effect
  • Only raised after the inner state is persisted by the aggregate.

Example

Let’s take an all-time favorite example of a shopping cart in Sales bounded context. Following are some of the key events identified by the team:

  • Cart Created
  • Cart Abandoned
  • Cart Submitted
  • Item Added to Cart
  • Item Removed from Cart

Now, when this was discussed in more detail to identify commands and policies, the team discovered a business rule that prohibits essential items to be more than the allowed quantity. If a customer exceeds the limit, then items are rejected by the cart and instead, it will be added it to the wish list. The wishlist will provide easy access in case the customer wants to swap it with something in the cart. This business rule is internal to the Sales bounded context.

Event Storming – Shopping Cart

Cart and Wishlist have close interactions and currently business has no use cases for Wishlist except for cart. With this in mind, the team has decided to keep the wishlist under the cart aggregate, therefore the same bounded context i.e. Sales. There is another bounded context that is touched upon in the above diagram, it is called ‘Order’.

Bounded Contexts – Events on the outside vs Events on the Inside

The team realised that not all these events are required outside, so by applying “Data on the Outside vs Data on the Inside” on events, the team tried to identify the events that are required outside the Sales bounded context:

The interactions within the Sales bounded context can be transactional i.e. Rejecting cart items and adding wishlist will be a part of the same transaction scope. However, generating orders when the cart is submitted will be an asynchronous operation and can be implemented as an eventually consistent state.

EventInsideOutside
Card Createdyesno
Cart Item Rejectedyesno
Cart Item Addedyesno
Wishlist Item Addedyesno
Wishlist Item Removedyesno
Cart Submittedyesyes

Domain Events

By this categorisation of the events, we unshackle the evolution of events on the inside. Now internally these events can be implemented in few different ways or some mixture of them:

  • Inter-Aggregate in-memory/in-process events
    • Go: Channels
  • Event Sourcing – Cart aggregate will construct the domain model from the events persisted in local state
  • CQRS – By using a message broker, cart aggregate will emit domain events and separate workers will maintain projections and/or read models by listening to the domain events.

In Memory Events
Following diagram illustrates how in-memory event publishers and subscribers can be used by application code to use domain events to communicate:

In Memory Events

The above diagram is an example of dotnet application using MediatR to facilitate inter-component, in-process communication. This provides flexibility to the developers to structure the code in a more modular way. Inter-aggregate domain events are a perfect fit for it. Because the publishing and consumption of events are in the same process, it allows the whole operation to be within the same transaction boundary. So if any of the subsequent event handlings fails (add wishlist), it would fail the whole operation.

Event Sourcing

Another use case of the domain events is in the event sourcing, whereby an aggregate saves to state in form of a sequence of various domain event-triggered to handle a command.

As shown in the diagram above, cart aggregate would accept the command and then load the current state before deciding the next action. The next actions would be stored as events in the database. Again, this whole process will be within a single transaction scope.

Integration Events

To intentionally keep the example easy, the only integration event we are looking at is the “Cart Submitted”. We know that the one or more external systems are interested in this event. Its “then”/”outside” characteristics mean, we should consider some of the above mentioned constraints or guidelines. The most important attribute is the immutability and read only nature of it. The recommendation is to provide a version on the event body.

CartSubmitted: {
  CartId: xxxxx,
  Version: 1,
  Customer: xxxxx,
  TotalAmount: $$,
  Items: []
}

This event would also carry the complete state to describe the event, so that the consumer would not need to come back. And if for any reason consumer application has to communicate back, then it will have the cart id and the version to refer to. This saves from the typical problem of the state updated after event was published. The version can also be supplied in the commands from external systems, this would allow the domain code to quickly reject the command if it is based on an out of date copy of the integration event.

Now if the business allows the cart to be edited event after it is submitted, then the order API knows to abandon the previous order when a new version of the cart submitted event is received. It may not be that straightforward, in case the order has since been processed already. There can be various ways to handle the situation based on the complexity of the actual business requirements, such as Workflow, Sagas, Microservices orchestration, etc. These patterns are not in scope of this post, one can write a book explaining these in details.

Conclusion

Event driven systems inherits many complexities of the integration of distributed systems, by applying some of the learnings from the standard integration patterns, we can limit the complexities to the events that truly need to be exposed outside of the domain. This provides the development team more flexibility to let the domain (inside) events evolve without any considerations of side effects on the external system. Similarly, by being aware of the integration events nature as “Data on the Outside”, we can apply some of the concepts like uniquely identifying every event (id + version). Accepting it’s “then” nature we make it explicit and build systems accordingly.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s