Undoing Half a Workflow: A Ledger for Testing Compensating Steps
When an integration performs several steps across different systems, there is no single undo. If step three fails, steps one and two have already happened. A compensating action is a separate step that tries to reverse an earlier one, and it can fail too.
This article proposes a saga step and compensation outcome ledger, and a way to rehearse it. It is a design suggestion for a test environment, not a description of anything already built.
Order the steps by reversibility
First sort the steps from most reversible to least. Put the hardest to undo last. In a hypothetical booking flow:
- Hold a calendar slot (can be released).
- Create a draft invoice record (can be voided inside your own system).
- Send a confirmation email (cannot be taken back).
Sending last means the irreversible action only happens once the others have succeeded.
The ledger
For each run, record a row per step with one of these outcomes.
| Outcome | Meaning |
|---|---|
| done | Step confirmed successful |
| failed_known | The other system clearly refused or errored before acting |
| unknown | No clear answer, such as a timeout |
| compensated | The reversal was confirmed |
| compensation_unknown | The reversal had no clear answer |
The difference between failed_known and unknown drives everything. A known failure allows compensation of earlier steps. An unknown outcome does not allow an automatic repeat of that write, because the first attempt may have taken effect. A local claim marker cannot prove otherwise.
Decision rules
- failed_known at step N: compensate steps 1 to N minus 1 in reverse order, recording each result.
- unknown at step N: stop, do not repeat step N, do not compensate yet, and ask for a check of the other system.
- compensation_unknown: stop, flag for a person, do not repeat the reversal.
Worked hypothetical
Three rehearsal runs, each injecting one fault in a test setup.
Run A: the invoice step returns a clear refusal. Ledger: step 1 done then compensated, step 2 failed_known, step 3 never started. Result: slot released, no email sent.
Run B: the email step times out. Ledger: step 1 done, step 2 done, step 3 unknown. Result: nothing is repeated or reversed. A person checks whether the email left. If it did, steps 1 and 2 stay. If it did not, the person decides between a manual resend and compensation.
Run C: the invoice step refuses, and then the slot release times out. Ledger: step 1 done then compensation_unknown, step 2 failed_known. Result: a person checks whether the slot is still held.
Counts: 3 runs, 1 fully compensated, 2 stopped for human checking. Check that each run has exactly one row per started step.
Stop conditions
Halt new runs of the workflow when:
- Several runs end in unknown or compensation_unknown within a short period.
- A compensation would affect something created by another run.
- The ledger cannot be written, since then you cannot trust what you did.
Proposed acceptance tests
These are proposals to try only against test systems, and none has been run.
- Run A, B and C above. Expect the ledger rows described, and no outbound request beyond the planned ones.
- During Run B, confirm that no retry is issued after the timeout, even if retries are enabled elsewhere.
- Replay the same run identifier. Expect the system to read the ledger and refuse to start new work.
- Interrupt the process between compensation steps. Expect the ledger to show which were confirmed.
- Ask a reviewer to explain, from the ledger alone, what state each external system is probably in and what is still unverified.
Limits
Compensation is not a perfect rewind. Some effects, such as an email already read, cannot be reversed, and some systems may expose no way to confirm a reversal. Treat the ledger as a record of what you know, and keep unknown as a legitimate result rather than forcing it into success or failure.