Cross-border payments are a sequence of steps that happen at different speeds, controlled by different parties, any of which can fail independently and at the worst possible moment. A simple request-response model fights that reality at every turn. An event-driven model embraces it. After building payment systems both ways, in anger and at scale, I will not go back, and this primer is the explanation I give engineers joining the team.
Why payments are naturally event-driven
A single cross-border transfer might involve initiation, compliance screening, foreign-exchange conversion, settlement with a banking partner in another country, and final confirmation back to the customer. These steps do not happen instantly, and they certainly do not happen in a tidy synchronous line. Compliance screening might take milliseconds or might take a human review measured in hours. A partner bank settles on its own schedule, not yours.
Modeling each step as an event, PaymentInitiated, ComplianceCleared, FxLocked, SettlementConfirmed, matches how the money actually moves through the real world. The architecture stops fighting the domain and starts mirroring it, and an enormous amount of accidental complexity simply evaporates.
The state machine underneath
Events without a state machine are just chaos with a message broker attached. Every payment is a state machine, and the events are the transitions between states. The discipline that makes this safe is to make illegal transitions impossible to represent. A payment cannot move to settled before compliance has cleared, and your code should refuse to even construct that state, not merely check for it later and hope.
Model the states explicitly and exhaustively. The bugs that genuinely scare me are not the ones that crash; they are the ones where money quietly ends up in a state the designers never imagined could exist.
I represent the state machine in code as an explicit enum of states and a single function that decides which transitions are legal from each one. Everything else routes through that function, so there is exactly one place in the system that knows the rules, and there is no path around it.
Idempotency and ordering
In any real message broker, messages get redelivered and can arrive out of order. This is not an edge case to handle someday; it is the normal operating condition, and a design that assumes exactly-once, in-order delivery is a design that will corrupt money in production. Two rules keep this survivable.
- Every consumer must be idempotent: processing the same event twice produces exactly the same result as processing it once.
- Never assume ordering. Carry enough state in the event or check enough state in the store to reject a transition that does not make sense yet.
If a SettlementConfirmed event somehow arrives before ComplianceCleared due to a redelivery quirk, the state machine rejects it because settling an uncleared payment is an illegal transition. The system stays correct even when the message bus misbehaves, which it will.
The audit trail you get for free
Here is the benefit that is underappreciated outside of regulated industries. An event log is, by its nature, an audit log. When a regulator or a customer or your own finance team asks what happened to a specific payment, you can replay the exact sequence of events with timestamps and reconstruct the truth precisely.
In a regulated business this built-in traceability is worth as much as the architectural flexibility, sometimes more. I have sat in audits where the difference between a short conversation and a painful one was entirely down to being able to say "here is the complete, ordered, timestamped history of this transaction" and mean it.
Where it costs you
I would be lying if I sold this as free. Event-driven systems are harder to reason about locally. You trade the comforting simplicity of a single call stack you can step through in a debugger for the resilience of decoupled steps you have to trace across services. Debugging spans multiple components. Eventual consistency genuinely confuses newcomers, who expect a write to be immediately visible everywhere. And you need real, sustained investment in observability or you will be blind.
For a simple CRUD application this trade is not worth it, and reaching for events there is resume-driven development. For money that crosses borders, partners, and time zones, it is the only model that has held up for me under real load and real failure.
The observability requirement
Because you have given up the call stack, you must buy back visibility deliberately. Every event carries a correlation ID that ties the whole payment together across services. Every consumer logs the transition it performed with that ID. A single dashboard can then show the complete life of any payment as a sequence of events, which is the thing that makes the architecture debuggable rather than merely elegant. Skip this, and event-driven design becomes a way to distribute your bugs across more services than you can hold in your head.
Conclusion: match the architecture to the domain
Cross-border payments are asynchronous, multi-party, and failure-prone by their fundamental nature. An event-driven architecture, anchored by an explicit state machine, made idempotent against redelivery, and backed by serious observability, matches that reality instead of fighting it. It costs you local simplicity and buys you resilience, auditability, and a system that tells the truth about what happened to every shilling that passed through it. For this domain, that is the right trade, every time.