Idempotency

An operation is idempotent if performing it more than once produces the same result as performing it exactly once. SET balance = 650 is idempotent — run it a thousand times and the balance is still 650. balance = balance + 50 is not — run it twice and it's 700. That distinction sounds academic until you notice how much of ordinary application code is built out of the second kind: charge the card, ship the order, send the notification, bump the counter. None of those are naturally safe to repeat, and networks repeat things constantly, whether you asked them to or not.

The Problem It Solves

A request can fail in two places: on the way there, or on the way back. If it fails on the way there, the server never saw it, and retrying is exactly correct. If it fails on the way back — the server did the work, committed it, and the acknowledgment got lost to a timeout, a dropped connection, a proxy restart — then the client is in an unresolvable position. It cannot distinguish "nothing happened" from "something happened and I just didn't hear about it." The only client-side options are give up (and risk a missed operation) or retry (and risk a duplicate one). Most client code, reasonably, retries.

This is not a hypothetical edge case; it is the normal operating condition of any system that talks over a network. Mobile clients on flaky connections, load balancers that kill long-running requests, message queues that redeliver on an unacknowledged message, a user who double-clicks "Submit" because the page didn't visibly respond fast enough — they all produce the exact same shape of problem: the same logical request arriving at the server more than once. Idempotency is what makes that arrival harmless.

A Worked Example: Bumping an Account Balance

Take an endpoint that adds a fixed amount to an account balance — a refund, a loyalty-point award, a wallet top-up. Expressed as SQL it's about as simple as an operation gets:

-- the risky version: relative, not idempotent
UPDATE accounts SET balance = balance + 50 WHERE account_id = 42;

Run it once, correct. Run it twice because a retry fired after a timeout, and the account is quietly overcredited by $50 with no error, no exception, and nothing in the logs that looks wrong in isolation — just a balance that's off. This is the exact shape of bug that survives code review, passes every unit test written against a single call, and only shows up in production, intermittently, under real network conditions.

Designing the Key

The fix is to have the client generate a unique token — an idempotency key — once per logical action, and attach it to every attempt at that action, retries included. The server records which keys it has already completed and what it returned, so a repeat arrival short-circuits into a replay instead of a re-execution:

CREATE TABLE idempotency_keys (
    idempotency_key      VARCHAR(64)  NOT NULL,
    account_id           BIGINT       NOT NULL,
    request_fingerprint   CHAR(64)    NOT NULL,  # sha256 of the request body
    status                ENUM('in_progress','completed') NOT NULL,
    response_body         TEXT,
    created_at            DATETIME    NOT NULL,
    PRIMARY KEY (idempotency_key, account_id)
);

The primary key does the concurrency work for free: two requests racing in with the same key will have one INSERT succeed and one fail on the duplicate-key constraint, so only one of them ever reaches the balance update. That's a stronger guarantee than an application-level "check then act," which has a race window between the check and the act.

Server-Side Flow

on POST /accounts/{id}/bump, header Idempotency-Key = $key:

  try to INSERT ($key, $id, fingerprint($body), 'in_progress', now())

  if insert failed on duplicate key:
    row = SELECT * WHERE idempotency_key = $key AND account_id = $id
    if row.request_fingerprint != fingerprint($body):
      return 409 Conflict   # same key, different payload — a client bug
    if row.status == 'completed':
      return row.response_body      # exact replay, no side effect
    return 409 Conflict          # still in flight, don't race it

  begin transaction
    UPDATE accounts SET balance = balance + $amount WHERE account_id = $id
    UPDATE idempotency_keys SET status = 'completed', response_body = $result
  commit

  return $result

The row insert and the balance update happen in the same commit boundary, so there is no window where the key is marked complete but the bump didn't happen, or vice versa. Storing the actual response body — not just a completed flag — matters: a retrying client should get back the exact same answer it would have gotten the first time, including any computed fields like fees or a resulting balance, not a generic "already processed" message it has to special-case.

Missed Posts: The Other Half of the Problem

An idempotency key protects against double posting. It does nothing at all about missed posting — if the client crashes, the process dies, or the caller simply gives up before ever retrying, the bump never happens, key or no key. These are two separate reliability problems that get solved by two separate mechanisms, and it's a mistake to expect one to cover the other:

• Double posting is solved by idempotency keys plus a lookup before every write.
• Missed posting is solved by durable retry — the caller (or a queue sitting in front of the caller) must keep the request around and keep trying until it gets a definitive answer, not give up silently on the first failure.

Put together, "durably retry until success" plus "idempotent on the receiving end" is what gets you to effectively-once processing — which is the practical, achievable target. True exactly-once delivery across an unreliable network is provably impossible to guarantee in the general case; effectively-once, where duplicates can arrive but are neutralized on arrival, is the achievable substitute and is good enough for almost everything in practice.

Things Worth Keeping In Mind

A few points that don't show up until you've actually built one of these, and that are easy to get subtly wrong even once you know the pattern:

The key has to be generated once per logical action, not once per HTTP attempt. The single most common mistake is a client library that mints a fresh UUID inside its retry loop — which defeats the entire mechanism, since every retry now looks like a brand-new request. The key belongs at the point where the user's intent is formed (the button press, the "place order" call), and every retry of that same intent must carry the same key forward.

Fingerprint the payload, not just the key. A key collision with a different request body is not a duplicate — it's either a client bug (a cached or reused UUID) or, worse, a key-guessing attempt. Comparing a hash of the body and rejecting a mismatch with a 409 is cheap insurance against both, and it's the detail most naive implementations skip.

Keys need a retention policy, not infinite life. Keep them long enough to cover the realistic retry window of your slowest client — 24 to 48 hours is a common choice for anything money-adjacent — then expire and purge them. Keeping them forever just grows an unbounded table for no benefit; expiring them too early reopens the double-processing window for a client that legitimately took a while to retry.

Not every operation needs this machinery in the first place. An assignment (set balance = 650) is idempotent by construction — replay it as many times as you want, the result never changes. Idempotency keys are specifically the tool for the operations that are inherently relative — bump, append, decrement, charge — where the naive version genuinely isn't safe to repeat. If you control both sides of an API, it's sometimes worth asking whether the operation can be redesigned as an assignment instead of bolted onto with a key. That's a real trade, though, not a free win — it just moves the race condition to whoever computes the target value client-side, and now clock skew and stale reads become the new failure mode.

Use a header, not a body field. Putting the key in an Idempotency-Key request header, following the convention Stripe popularized, keeps "what should happen" (the body) cleanly separate from "how many times am I allowed to ask for this" (the header). It also makes the key trivially easy to grep out of logs and traces independent of whatever the body schema looks like this API version.

An idempotency table is itself a small piece of durable state you now have to operate — it needs a primary key, an index your write path actually hits on every request, and a purge job someone remembers exists. That's a real, ongoing cost. It's a cost worth paying for anything that moves money, inventory, or an irreversible side effect, and probably not worth paying for a request that's naturally idempotent already or whose worst failure mode is genuinely harmless to repeat. The mechanism is cheap per request; it is not free to own.

Further Reading

• Wikipedia: Idempotence — the general mathematical and computing definition, beyond the API-request framing used here.
• Stripe: Idempotent Requests — a widely-imitated, production-proven implementation of the header-based key pattern.
• IETF Draft: The Idempotency-Key HTTP Header Field — the standardization effort formalizing the convention across the industry.
• Wikipedia: Two Generals' Problem — the underlying reason acknowledgment over an unreliable channel can never be made fully certain.

Checklist

Recognizing the risk:
  [ ] Identify every endpoint that mutates state relative to its current value (bump, charge, ship, notify)
  [ ] Assume any such request can and will arrive more than once over a real network
  [ ] Distinguish "server never saw it" from "server did it and the ack got lost" — only the second needs a key

Designing the key:
  [ ] Generate the key once per logical action, client-side, and reuse it across every retry of that action
  [ ] Scope the key to account/endpoint so unrelated operations can't collide
  [ ] Store a fingerprint of the request body alongside the key and reject mismatches
  [ ] Store the actual response, not just a completed flag, so replays are indistinguishable from originals
  [ ] Let the primary-key constraint — not an application-level check-then-act — resolve concurrent duplicates

Operating it:
  [ ] Commit the key's status and the state change in the same transaction
  [ ] Set a retention window and purge expired keys on a schedule
  [ ] Pair the key with durable client-side retry — a key alone doesn't stop a missed request from staying missed
  [ ] Skip the machinery for operations that are already naturally idempotent (absolute assignments)