In response to the growing demands of the modern market, Organizations or Businesses today begin their digital journey with MicroServices Architecture or migrating their applications architecture from Monolithic to Microservices in order to have faster Time to Market, improved Customer Satisfaction, Development Productivity, and reduced Cost-to-Maintain. 

Although MicroServices Architecture brings in lots of benefits, it also comes with its own challenges. One such challenge is Distributed Transaction, a Transaction that spans multiple services.

Before understanding the distributed transaction, let’s understand the basics of transaction and how it works well in Monolithic Applications but stands a challenge in the Microservices World. 

Transaction Fundamentals

Transaction in the context of databases or storage systems is an operation that is treated as a single unit of work, which either completes fully or does not complete at all and ensures a consistent state of the storage system. 

A Transaction is described by four characteristics, which together are termed as an ACID:

• Atomicity All operations are executed successfully or everything fails together.
• Consistency The data in the database is kept in a valid state.
• Isolation Separated transactions running concurrently cannot interfere with each other.
• Durability After a transaction is committed, the changes are stored durably, e.g. persisted in a disk. 

In a Monolithic World, as transactions do occur within a service boundary, they can be committed or rollbacked effortlessly. 

Let’s talk about distributed transactions in the MicroServices World

In MicroServices Architecture, an application is composed of a smaller set of services in order to have independent deployability, scalability, and loose coupling. As per the Database-per-service pattern, individual services should have their own database in order to achieve independent deployability, scalability, and loose coupling requirements. 

Since each service has its own persistent store, business transactions can span across multiple services which are composed of multiple local transactions within individual services. In order to have successful business transactions, either all local transactions succeed or roll back completely, in case any of the local transactions fail in order to ensure data integrity.   

Solutions in order to implement Distributed Transactions:

1) Two-Phase Commit (2PC)

2) SAGA Design Pattern (Eventual Consistency and Compensation) 

Two-Phase Commit (2PC)

One of the important participants in a distributed transaction is the transaction coordinator.

The distributed transaction consists of two steps:

• Prepare phase — During this phase, all participants of the transaction prepare for commit and notify the coordinator that they are ready to complete the transaction
• Commit or Rollback phase — During this phase, either a commit or a rollback command is issued by the transaction coordinator to all participants 

Two-phase commit challenges:

• Coordinator node can become the single point of failure
• Slow by design due to dependency on the coordinator and
• Protocol not supported in NoSQL databases. 

SAGA Design Pattern (Eventual Consistency and Compensation)

A saga is a sequence of local transactions. Each local transaction updates the database and publishes an event to trigger the next local transaction in the saga. In the saga, each local transaction has a corresponding compensating transaction, in order to roll back, its effect. If a local transaction fails then the saga executes a series of compensating transactions that undo the changes that were made by the preceding local transactions.

In the Saga pattern, a compensating transaction must be idempotent and retryable. 

Two ways of Saga Coordination

Choreography – each local transaction publishes events that trigger local transactions in other services

Orchestration – an orchestrator service responsible for triggering local transactions within domain services and publishing events across services in order to complete or roll back a business transaction. 

Choreography oriented SAGA

In Choreography-based SAGA, each participating microservice within a business transaction publishes events to the next microservice. Choreography-based flow is considered successful if all the participating microservice completes their local transaction without failure. 

Choreography based SAGA can further be implemented in two ways based on how events are being used: 

Choreography using Outbox Store (Outbox Pattern)
Choreography using Event Sourcing (Event Sourcing Pattern)

Choreography using Outbox Store

Each service needs to maintain a separate table (outbox table) corresponding to each table that falls under the local transaction. It involves a two-step transaction, one to the original table and the other to the corresponding outbox table. A separate Message Relay process then publishes the events from the outbox table to a message broker, which needs to be polled by the next participating microservice. 

To overcome the drawback of continuous polling, we can also make use of the tool Debezium, which can perform change data capture (CDC) using Service A database transaction log and publish to message broker. In this way, Service B can listen to generic events in a topic rather than polling Service A’s database.

Choreography using Event Sourcing 

In this approach, instead of using the traditional CRUD-based model, the application state is being stored in the form of events in the Event Store. It is an approach where we can get all the application state changes as a sequence of events. Event Store provides an API that enables services to subscribe to events.

Here, service A publishes its state change event to Event Store as part of the local transaction, and Service B being a subscriber to Event Store, receives the published event and thereby completes its local transaction which ultimately results in a successful business transaction. 

Advantages 

• No central transaction coordinator.
• Improved scalability and resilience characteristics.

Disadvantages

• Global system state and coordination logic is scattered across all participating microservices.
• Eventual Consistency

Orchestration oriented SAGA

In Orchestration-based SAGA, the participating microservices do not directly interact with each other. Instead of that, we have a common orchestrator service or engine which is responsible for managing the overall transaction status. If any of the microservice local transactions fails, then the orchestrator is responsible for invoking the necessary compensating transactions.

The jBPM, Camunda, Serverless stateful functions such as Amazon StepFunctions, Azure Durable Functions, Azure Logic Apps, and Google Workflows are the popular implementations of the orchestration technique.

Advantages 

• Coordinates state among heterogeneous distributed components.
• Known distributed state at the coordinator level.

Disadvantages

• Complex distributed programming model.
• Eventual Consistency

Conclusion

In this article, we discussed the transaction fundamentals and how it turns out to be a challenge in the Microservices World. We then talked about a two-phase commit as a solution and examined its shortcomings with better alternatives. 

We realized that Saga Pattern is a go to approach when distributed transactions need to be carried out in the Microservices World.

Hence Choreography Saga is suitable for greenfield microservice application development and Orchestration Saga is useful for brownfield microservice application development architecture.