Buying motor insurance online looks like one click. Behind it, after the payment goes through, the system still has to confirm the final price with the insurer, lock that quote, create the policy on the insurer's side, store the policy document and update everything the user sees. Several of those steps call an external API that is sometimes slow and sometimes down.
The worst outcome is not an error. It is a user who has paid and has no policy, or a policy created twice. These are the patterns I used to rule both out.
Break issuance into a chain of queue steps
Instead of one long request that does everything, issuance runs as a sequence of small steps, each triggered by a message on its own queue. Each worker does one thing, records that it is done, and publishes the message for the next step.
This buys three things at once:
- Resumability. If step five fails, steps one to four are not repeated. The retry starts at step five with everything it needs in the message and the shared application record.
- Isolation. A slow insurer slows the steps that call it, not the whole pipeline or the user-facing API.
- Visibility. Queue depth per step tells you exactly where things are stuck.
Make every step idempotent
Queues deliver at least once, so every step will sometimes run twice. Each worker first checks a record of the workflow's current step and status: if this step already completed, it acknowledges the message and does nothing. Calls to the insurer carry an idempotency key, so a retried "create policy" returns the policy that already exists instead of creating a second one.
Retries are only safe when doing something twice has the same effect as doing it once.
Retries, then a dead-letter queue
Transient failures, such as a timeout or a 503, are retried with backoff. After a fixed number of attempts, the message moves to a dead-letter queue for that step. Nothing is silently dropped: an engineer can inspect the message, fix the cause and redrive it, and the chain carries on from there. Insurer calls also sit behind circuit breakers, so a partner that is down fails fast instead of tying up every worker.
Re-check the price before you commit
A quote can change between the moment the user picks it and the moment they pay. So the pipeline fetches the revised quote first and only continues if it matches what the user approved, then locks it with the insurer before creating the policy. It is one extra call, and it prevents the most expensive class of support ticket.
Keep shared state in one place
Every service in the journey reads and writes one versioned application record rather than passing state point to point. Updates use optimistic locking on a version number, so two concurrent actions on the same purchase can't overwrite each other. The home screen reads the same record, so the user sees the new policy as soon as the last step finishes.
None of these ideas is new. Together they turn "the insurer timed out" from an incident into a retry nobody notices.