Assume dependencies fail
Models time out, networks disappear, applications close and permissions change. Reliability starts by treating these as normal operating conditions.
Bound retries
Retry only when the failure is plausibly transient and cap the number of attempts. Repeating the same failed action forever is not resilience.
Preserve enough state to recover
Long-running work should be able to resume from a safe checkpoint when that is appropriate for the product. Keep recovery data minimal and integrity-checked.
Use alternate routes carefully
Fallbacks should remain within the same user-approved scope. A blocked path must not silently become broader authority.
Expose failure honestly
When recovery is exhausted, report what succeeded, what failed and what the user can do next. Public guidance can explain these principles without publishing a commercial product's routing or recovery implementation.