GitHub Timed Out. Did the Review Still Get Posted?

Imagine a code review finishes, Scopebase sends its comments to GitHub, and the request times out before the response comes back.

Did GitHub create the review?

It might have. GitHub could accept the request, save the review, and lose the response somewhere on its way back. Or it might never have received the request at all. From Scopebase's point of view, those two histories look exactly the same.

The obvious move is to retry. Until the first request turns out to have worked. Then the pull request has two reviews, and the developer has to decide which one to read.

That small failure is one of my favorite architecture problems because there is no clever timeout value that fixes it. The caller has to reason about an outcome it cannot directly observe.

The gap between the write and the reply

People often use “retry” and “idempotent” as if they make external requests safe by default. They don't. A request that creates a review changes state in another system. If GitHub processed it, repeating it can create that state twice.

Scopebase's database and GitHub do not share a transaction. Scopebase can't commit “review created” locally at the exact instant GitHub commits the review remotely. There's always a gap between sending the request and learning what happened.

So I stopped asking “Did the request succeed?” and started asking “What evidence would let recovery adopt the result if it did?”

Save intent before sending

Before the create request goes out, Scopebase saves the delivery intent and moves the operation to MAY_HAVE_BEEN_SENT.

That state name is intentionally nervous. It says the remote write may have happened, so a restart cannot quietly treat it as a brand-new operation. Scopebase also attaches a marker tied to the run and revision, and fingerprints the sanitized comment content. These are clues for recovery. They do not make GitHub enforce an idempotency key.

The write-ahead intent handles the crash that happens after Scopebase sends but before it records the response. After a restart, the worker sees that the operation may have been sent and knows to reconcile before trying again.

Recovery needs more than matching text

When the response is lost, Scopebase looks for an existing review. It checks the expected bot actor, submitted state, commit, run-and-revision marker, and fetched comment content against the saved fingerprint.

Why check all of that? Because “I found a review with similar text” is weak evidence. The author could be different. The review could belong to another commit. The comment content could have changed. Recovery should adopt the artifact only when it can tie it to this exact operation.

If the checks match, Scopebase adopts the existing review and records delivery as complete. The model does not run again. The analysis is already retained separately from delivery, and repeating a paid model call would not answer whether GitHub created the review.

If a bounded search finds nothing, Scopebase still does not declare the request failed. The review could be outside the search result, or the search may not prove absence. The operation remains uncertain rather than returning to “not started.”

That distinction matters: failure to find evidence is not evidence that nothing happened.

This gives up some availability

Here's the uncomfortable case. Scopebase saves MAY_HAVE_BEEN_SENT, then crashes just before opening the network connection. After restart, that local state looks just like a request GitHub accepted while its response disappeared.

The system can't safely distinguish those histories. It may leave a review undelivered until controlled recovery resolves the ambiguity, even though GitHub never saw the request.

I accepted that cost to avoid automatically creating duplicate reviews. This is a trade between availability and duplicate side effects. It is not “exactly once.” Without GitHub participating in the same transaction or providing an idempotency guarantee for this operation, Scopebase cannot promise exactly one remote write.

It can make uncertainty durable, look for strong evidence, and avoid blindly repeating an operation that may have succeeded. That's smaller than exactly-once delivery, and it's a promise I can defend.

The same question shows up elsewhere

This failure isn't specific to code review. A payment can go through while the checkout request times out. A deployment can start while the dashboard loses its response. A webhook-triggered action can succeed before the sender gives up waiting.

In each case, the useful question is not simply “Should I retry?” It's “What evidence would prove that a retry is safe?”

For Scopebase, the answer is a saved operation marker plus matching actor, commit, and comment content. When those checks don't settle it, the system stops and keeps the unknown outcome visible.

I like that answer because it doesn't pretend the network is more reliable than it is. A timeout isn't a verdict. It's a question the system needs to carry forward.