Lesson 19 · Distributed transactions · Module 2
Giving Up Atomicity
If you will not pay 2PC's blocking tax, the only other move is to stop asking for one atomic outcome and start asking for a sequence of small ones, each with a defined way to be taken back. The bill arrives in a different currency: everything in the middle becomes visible, and an undo is now a new fact rather than the erasure of an old one.
The win in this lesson: you will be able to take a multi-step business operation, write the compensation for each step, and say precisely what becomes observable that a single transaction would have hidden.
1. The operation that was never going to be one transaction
Lesson 18 ended on a refusal: across two services, in two regions, you almost never want a protocol whose worst case is healthy participants holding locks until a human arrives. But the business operation does not go away. Reserve the stock, charge the card, book the courier, send the confirmation. Four steps, four systems, minutes of wall-clock time.
The 1987 paper that named the alternative was not thinking about microservices at all. It was thinking about transactions that simply run for a long time, and what they do to everyone else:
"Long lived transactions (LLTs) hold on to database resources for relatively long periods of time, slgmficantly delaymg the termmatlon of shorter and more common transactions"
Garcia-Molina and Salem, SAGAS (ACM SIGMOD, 1987), Abstract
Swap "holds locks for hours" for "holds locks across a network call to a payment provider" and it is the same problem, with a worse blast radius. The paper's answer is to stop treating the long operation as atomic at all.
2. The mechanism
"A LLT 1s a saga if it can be written as a sequence of transactions that can be interleaved with other transactions The database manage-ment system guarantees that either all the tran-sactions m a saga are successfully completed or compensatmg transactions are run to amend apartial execution"
Garcia-Molina and Salem, SAGAS, Abstract
Three commitments in one sentence, and it is worth separating them:
- Each step is a real transaction that commits on its own. No step waits for another.
- Each step has a compensating transaction defined in advance, written by you.
- The guarantee is all steps, or a prefix of steps plus their compensations run in reverse — never a half-done prefix left lying there.
And the definition of the compensation is the sentence to memorise:
"The compensatmg transaction undoes, from a semantic point of view, any of the actions performed by T, […] but does not necessarily return the database to the state that existed […]"
Garcia-Molina and Salem, SAGAS, §1 Introduction
From a semantic point of view is doing all the work. A rollback restores bytes. A compensation restores meaning, by writing more bytes.
3. What you gave up: isolation
Not durability, not per-step atomicity. Isolation. The paper says so in a parenthesis, which is where the expensive properties of a design usually hide:
"Note that other transactions might see the effects of a par-tial saga execution […] no effort 1s made to notify or abort transactions that nught have seen the results of T, before they were compensated for"
Garcia-Molina and Salem, SAGAS, §1 Introduction
Read the second half again. It is not only that readers can see the intermediate state — it is that when you compensate, nobody tells them. Anything they did on the strength of what they saw is now your problem, and the system does not know it happened.
2PC hides the intermediate state and pays for it with a blocking window. A saga eliminates the blocking window and pays for it by publishing the intermediate state. You do not get to avoid the middle of a multi-step operation. You only get to choose whether it is invisible and fragile, or visible and durable.
4. Designing the compensations
The paper is honest that this is the hard part, and that some steps have no honest compensation at all:
"Designing compensatmg transactions for LLTs is a difficult problem rn general (For instance, if a transaction fires a missile, it may not be possible to undo this action )"
Garcia-Molina and Salem, SAGAS, §9 Designing Sagas
But it also gives the move you will actually use for the awkward ones — compensate in the world, not in the database:
"It may even be possible to compensate for actions that are harder to undo, like sending aletter or prmtmg a check For example, to com-pensate for the letter, send a second letter explaining the problem To compensate for the check, send a stop-payment message to the bank"
Garcia-Molina and Salem, SAGAS, §9 Designing Sagas
Four steps from the saga above, written out as they would actually go in a design document:
| Step | Compensation | What stays observable afterwards | Data it needs |
|---|---|---|---|
| Reserve stock | Release the reservation | A reservation row with status released; the count dipped and recovered |
Reservation id, captured at step time |
| Charge card | Issue a refund | Two entries on the customer's statement, days apart, both real | Provider charge id — not the order id |
| Allocate an account number | Deactivate it; never reissue | A permanent gap in the sequence, which some report will ask about | The allocated number |
| Send confirmation email | None. Best available: send a second email explaining | The first email, in an inbox, forever | Recipient and the original message id |
Three rules fall out of that table, and they are the whole practice:
- Capture the compensation's arguments when the step runs, not when it fails. The provider's charge id exists only after the charge. Write it down in the same transaction that records the step, or the compensation has nothing to aim at.
- Every compensation must be idempotent and retryable. A crash mid-rollback re-runs it. This is Lesson 04's idempotence key at a new layer: refund keyed by charge id, not "refund the most recent charge".
- A compensation may not fail for business reasons. It cannot be declined, cannot require approval, cannot run out of stock. If it can, it is not a compensation, it is another step that will need one.
5. Ordering: put the irreversible steps last
Because compensations run backwards from the failure point, the order of your steps decides which failures are survivable. The shape you are aiming for splits the saga into three zones.
The paper's other design lever is subtler and easy to miss: if a step leaves the database in a state that makes no sense, the fix may be to give that state a name in the schema rather than to hide it.
"After T1 com-pletes, the database IS left m an mconslstent state because some money 1s “mlssmg,” 1 e , It cannot be found m the database […] we must incorporate mto the data-base schema the “temporary” storage (e g , we add a relation for funds m transit or for pendmg insurance claims) Also, transactlons that need to see all the money must be aware of this new storage"
Garcia-Molina and Salem, SAGAS, §9 Designing Sagas
That is the difference between a saga that works and one that produces mystery tickets. The money is not missing — it is in transit, and every report that adds up money must know that word exists. Intermediate states are visible; the design choice is whether they are also named.
6. Residual risk
A compensation can fail, and then nothing saves you. The paper reaches this point too, and does not pretend otherwise:
"The transaction could be aborted, but if it were run again it would probably encounter the same error In this case, the system is stuck it cannot abort the transaction nor can it complete it"
Garcia-Molina and Salem, SAGAS, §6 Other Errors
Stuck is stuck in both designs. But compare the cost of being stuck with the 2PC version from
Lesson 18, where the exit was a human reading
pg_prepared_xacts while locks piled up behind them:
"Fortunately, while the transaction 1s bemg manually repalred the saga does not hold any database resources (1 e , locks) Hence, the fact that an already long saga ~111 take even longer will not slgmficantly affect performance of other transactions"
Garcia-Molina and Salem, SAGAS, §6 Other Errors
That is the trade in one line. Both designs can end in a person fixing something by hand. Under 2PC the rest of the database waits for that person. Under a saga it does not — one order is wrong, and everyone else keeps trading.
Three more risks that are yours to carry:
- Somebody acted on the intermediate state. A report was emailed, a downstream system consumed an event, a customer saw the charge. Compensating does not retract any of it, and the paper guarantees no notification.
- The compensation window is a real duration. Between failure and compensation the money is taken and the order does not exist. Retrying the failed step politely for ten minutes before giving up means ten minutes in that state. Decide the deadline deliberately.
- Someone will call it a rollback. The moment a saga is described in review as "and then it rolls back", the team has stopped designing compensations and started assuming them. The refund is a new fact. The released stock is a new count. The email is still in the inbox.
7. Check yourself
8. Back to your world
Pick one multi-step operation in your own system — signup, checkout, provisioning, cancellation. Write the steps in order, and beside each one write its compensation and the data that compensation needs. You will find two things within ten minutes: a step whose compensation nobody has ever written, and a step that is irreversible but is not last.
Then ask the second question, the one teams skip: what does a customer, a report, or a downstream consumer see at each intermediate point? If the answer to any of them is "something we cannot explain", that state needs a name in the schema before it needs a fix in the code.