Write acceptance criteria for failures, retries, and partial success

The happy path is the easy part

Most acceptance criteria describe what happens when everything works. The user submits a form, the system processes it, a confirmation appears. Those criteria are necessary, but they cover the minority of real production behavior. The interesting -- and dangerous -- territory is what happens when a downstream service times out, a user submits twice because the button felt unresponsive, a permission gets revoked mid-flow, or a batch import succeeds for 940 rows and fails for 60.

When these scenarios lack explicit criteria, engineers fill the gaps with their own judgment. That judgment varies by person, by day, and by how much context they absorbed from a Slack thread three sprints ago. The result is inconsistent error handling, ambiguous user states, and support tickets that nobody can reproduce because the failure path was never specified.

This guide is for product managers writing stories that cross system boundaries, and for engineers reviewing those stories before implementation begins. It supplies a reusable worksheet, a worked hypothetical scenario, and a diagnostic table for the failure modes that appear after launch when edge-case criteria were skipped.

Quick-start path

If you need to improve your next story in fifteen minutes:

1. List every external dependency the feature touches (API, database, queue, third-party service, user session).
2. For each dependency, ask: what happens if it is slow, down, or returns something unexpected?
3. For each user action, ask: what happens if the user does it twice, does it with stale data, or loses authorization between start and finish?
4. Write one acceptance criterion per answer using the format in the worksheet below.
5. Tag each criterion as retryable, permanent-reject, or unknown-outcome.
6. Add the criteria to your coverage matrix and confirm each one maps to a test and a recovery action.

That sequence alone catches the majority of gaps. The rest of this guide explains why each step matters and how to handle the harder cases.

Map actors, state transitions, dependencies, and irreversible effects

Before writing any edge-case criteria, you need a clear picture of four things:

Actors

Who or what initiates and participates in the workflow? A user clicking a button is obvious. Less obvious: a background job retrying a failed webhook, a third-party callback arriving after the user has navigated away, an admin revoking access while a long-running process is in flight.

State transitions

Draw the states a record can pass through. For an import job, this might be: draft -> validating -> importing -> partial_success -> complete or draft -> validating -> failed. Every arrow between states is a place where something can go wrong. Pay special attention to transitions that cannot be reversed -- a sent email, a charged payment, a published record visible to end users.

External dependencies

List every system boundary the workflow crosses. Each boundary is a potential failure point with its own failure semantics. A PostgreSQL query failing is different from an HTTP call to a partner API timing out, which is different from a message disappearing from a queue.

Irreversible effects

Flag any action that cannot be undone programmatically. If a step sends a notification, charges money, or writes to an external system you do not control, the criteria for that step need special care. You cannot retry an irreversible action the same way you retry a database write.

Why this matters: Without this map, you will write criteria for some edge cases and miss others. The map does not need to be a formal diagram. A bullet list in the story is enough, as long as it is explicit. Teams using product planning tools can attach this dependency list directly to the feature spec so it stays linked to the criteria.

Adapting to team context

• Small team, single service: Your dependency list may be short (database, one external API, user session). Focus your criteria on the API boundary and on user-facing state after failure.
• Microservices or distributed system: Every inter-service call needs its own failure criteria. Pay particular attention to eventual consistency -- the user may see stale data from a read replica while a write is still propagating.
• Regulated domain (finance, health): Irreversible effects require audit trails. Your criteria should specify not just what happens on failure, but what gets logged and who gets notified.

Write criteria for the six common edge cases

Each criterion below follows a structure: Given (precondition), When (trigger, including the failure), Then (observable outcome for the user and the system). This is not decorative -- the structure forces you to specify what the user actually sees, not just what the system does internally.

1. Timeouts

A timeout is not a failure. It is an unknown outcome. The request may have succeeded on the other side. Criteria must distinguish between "we know it failed" and "we do not know."

Example criterion (hypothetical): Given a user has submitted a data import and the import service has not responded within 30 seconds, when the timeout is reached, then the UI displays "Import is still processing -- we will email you when it completes or fails," the import job is marked pending_confirmation in the database, and a reconciliation job checks the import service status every 60 seconds for up to 10 minutes.

2. Duplicate submission

Users double-click. Network retries fire. Queues deliver the same message twice. Criteria must specify idempotency expectations.

Example criterion (hypothetical): Given a user clicks "Submit" on an order form and the request is sent twice due to a network retry, when both requests arrive at the server, then only one order is created, the second request returns the same order ID as the first, and the user sees a single confirmation.

3. Stale data

A user opens a form, walks away, comes back, and submits. Meanwhile, the underlying record has been modified by someone else.

Example criterion (hypothetical): Given a user is editing a configuration record that was last modified at timestamp T1, and another user modifies the same record at timestamp T2 > T1, when the first user submits their edit, then the system rejects the submission with a conflict error, displays the current values alongside the user's attempted changes, and offers the user the option to overwrite or discard.

4. Authorization changes mid-flow

A user starts a multi-step workflow with valid permissions. An admin revokes those permissions before the user finishes.

Example criterion (hypothetical): Given a user has started the approval workflow and their "approver" role is revoked while they are on step 2 of 3, when they attempt to submit step 3, then the system returns a 403 error, displays "Your permissions have changed -- please contact your administrator," and does not save the partial approval as complete.

5. Partial success in batch operations

This is the most commonly under-specified edge case. A batch of N items is processed; some succeed and some fail.

Example criterion (hypothetical): Given a user uploads a CSV with 1,000 rows for import, and 940 rows pass validation while 60 rows contain invalid email addresses, when the import completes, then the 940 valid rows are imported, the 60 invalid rows are collected into a downloadable error file with row numbers and error descriptions, the UI shows "940 of 1,000 rows imported -- download error report for details," and the user can re-upload a corrected file containing only the failed rows.

6. Permanent rejection vs. retryable failure

Not all failures are equal. A 400 (bad request) is permanent -- retrying will not help. A 503 (service unavailable) is transient -- retrying might help. Criteria must tell the user which situation they are in.

Example criterion (hypothetical): Given a payment submission returns a "card declined" response from the payment processor, when the system receives this permanent rejection, then the user sees "Your card was declined -- please try a different payment method," no retry is attempted, and the order remains in payment_pending state for 24 hours before automatic cancellation.

When edge cases combine: compound failures and irreversible-effect timeouts

The six edge-case types above are useful individually, but production failures rarely arrive one at a time. The hardest criteria to write -- and the ones most often missing -- cover situations where two edge cases occur simultaneously, especially when one of them involves an irreversible effect.

The compound-failure problem

Consider a batch payment operation (hypothetical). A payroll system sends 200 payment instructions to a banking API. Payments 1-150 succeed and are confirmed. Payments 151-180 are rejected (insufficient funds in the source account after the first 150 drew it down). Payments 181-200 time out -- the banking API did not respond.

This is a partial success (edge case 5) combined with an unknown outcome (edge case 1), and some of the completed actions are irreversible (money has moved). You now face three populations of records in a single batch, each requiring different criteria:

• Confirmed (1-150): No action needed. But the user must see these clearly separated from the other two groups.
• Rejected (151-180): Permanent failure. The user needs to know why and what to do (add funds and resubmit, or split the batch).
• Unknown (181-200): The system cannot tell the user whether these payments went through. Retrying risks double-payment. Not retrying risks non-payment.

Example compound criterion (hypothetical): Given a payroll batch of 200 payments where 150 are confirmed, 30 are rejected, and 20 have timed out with no response from the banking API, when the batch summary is displayed, then the UI shows three groups: "150 completed," "30 rejected (see reasons)," and "20 awaiting confirmation -- do not resubmit these until status is resolved." The system schedules a reconciliation job that queries the banking API's transaction-status endpoint every 2 minutes for up to 30 minutes. If reconciliation confirms a payment succeeded, it moves to the "completed" group. If reconciliation confirms it failed, it moves to the "rejected" group with a "safe to retry" flag. If reconciliation cannot determine status after 30 minutes, the system alerts an operations team member and the user sees "20 payments require manual review -- our team has been notified."

The decision heuristic: when does "unknown" need active reconciliation?

Not every unknown outcome justifies a reconciliation job. The decision depends on two factors:

1. Is the effect irreversible? If a timed-out request might have sent money, published content, or triggered a notification that cannot be recalled, you need active reconciliation. Passive user notification ("try again later") is not safe because the user might retry and cause a duplicate irreversible effect.

2. Does the downstream system offer a status-check endpoint? If there is no way to query the outcome, reconciliation is impossible and your criteria must say so honestly. The criterion should specify a manual escalation path, not pretend the system can resolve the ambiguity automatically.

If both answers are yes -- irreversible and checkable -- write a reconciliation criterion. If the effect is reversible (e.g., a draft record that can be safely overwritten), passive notification with a user retry is sufficient. If the effect is irreversible but not checkable, your criterion must route to a human operator. Pretending otherwise creates false confidence.

This heuristic should be applied during the mapping step: for each irreversible effect you flagged, ask whether the external system exposes a status query. If it does not, note this in the coverage matrix under "Recovery Action" as "manual escalation required -- no status endpoint available." That honest gap in the matrix is more valuable than an automated-sounding recovery that cannot actually work.

Coverage matrix worksheet (copyable)

Copy this table into your story or spec. Fill one row per edge-case criterion. The instructions follow the table.

ACCEPTANCE CRITERIA COVERAGE MATRIX
====================================

Feature: [Name of feature or story]
Author: [PM or engineer name]
Date: [Date]
Dependencies: [List every external system, API, queue, database]

| ID   | Edge Case Type        | Given (Precondition)                  | When (Failure Trigger)                     | Then (User Outcome)                              | Then (System Outcome)                        | Failure Class       | Test Method           | Recovery Action                    |
|------|-----------------------|---------------------------------------|--------------------------------------------|--------------------------------------------------|----------------------------------------------|---------------------|-----------------------|------------------------------------|
| EC-1 | Timeout               | User submitted import; service slow   | Import service exceeds 30s timeout         | UI shows "still processing" + email promise      | Job marked pending_confirmation              | Unknown outcome     | Integration test      | Reconciliation job polls service   |
| EC-2 | Duplicate submission  | User clicks Submit on order form      | Two identical requests arrive              | Single confirmation, same order ID               | Only one order created (idempotency key)     | Retryable (safe)    | Load test / API test  | Deduplication via idempotency key  |
| EC-3 | Stale data            | User editing config, record changed   | Submit with outdated timestamp             | Conflict error with diff view                    | Write rejected, no data loss                 | Permanent rejection | Unit test + UI test   | User resolves conflict manually    |
| EC-4 | Auth change mid-flow  | User in multi-step flow, role revoked | Submit after permission revoked            | 403 + human-readable message                     | Partial work not saved as complete           | Permanent rejection | Integration test      | User contacts admin                |
| EC-5 | Partial success       | CSV with 1000 rows, 60 invalid        | Import completes with mixed results        | Summary + downloadable error file                | 940 rows committed, 60 rows in error table   | Mixed               | Integration test      | User re-uploads corrected rows     |
| EC-6 | Permanent rejection   | Payment submitted, card declined       | Processor returns decline code             | "Try different method" message                   | Order stays in payment_pending 24h           | Permanent rejection | Mock API test         | User retries with different card   |
| EC-7 | Compound: partial +   | Payroll batch, 20 payments timed out  | Timeout during irreversible batch          | Three-group summary; "do not resubmit" warning   | Reconciliation job queries status endpoint   | Mixed + Unknown     | Integration + mock    | Reconciliation -> manual escalation|
|      | unknown + irreversible|                                       |                                            |                                                  |                                              |                     |                       |                                    |

INSTRUCTIONS:
- Add one row for every edge case you identify. Do not limit yourself to the types above.
- "Failure Class" must be one of: Retryable, Permanent rejection, Unknown outcome, Mixed.
  For compound failures, use a combination (e.g., "Mixed + Unknown").
- "Test Method" should name the kind of test, not just "automated." Integration tests
  and mock-based tests catch different things.
- "Recovery Action" is what happens AFTER the failure -- who or what fixes the state.
  If the answer is "nothing," that is a gap. If the answer is "manual escalation
  required -- no status endpoint available," write that explicitly.
- For any row involving an irreversible effect, confirm that the recovery action
  accounts for the possibility that the action already completed on the remote side.
- Review this matrix with your engineering lead before implementation begins.
- Revisit after the first production incident involving this feature.

How to use this worksheet: Fill it out during story refinement, not after. Walk through it in your planning session. If an engineer cannot identify the recovery action for a row, that row needs design work before it needs code. Teams that organize planning with tools like FeatureFactory can link each row to the relevant story so the criteria survive sprint transitions.

Hypothetical worked example: a contact-import flow with partial success

This scenario is entirely hypothetical. All numbers, system names, and outcomes are illustrative.

Setup

A SaaS application lets users import contacts from a CSV file. The import flow touches three systems: the application server, a validation microservice, and an email-verification API provided by a third party. The user uploads a file, the system validates it, sends emails through the verification API, and writes valid contacts to the database.

What happens step by step

Step 1: Upload. The user uploads a 500-row CSV. The application server accepts the file, assigns it job ID import-7742, and returns a "processing" screen with a progress indicator.

Step 2: Validation. The validation microservice checks each row for required fields, format rules, and duplicates against existing contacts. 480 rows pass. 20 rows fail (12 missing a required field, 8 duplicating existing contacts). The 20 failures are written to an error log keyed to import-7742.

What the user sees: The progress bar advances. No failure is shown yet because the job is not complete.

Step 3: Email verification. The 480 valid rows are sent to the third-party email-verification API in batches of 100. Batches 1-4 (400 rows) return successfully. Batch 5 (80 rows) times out after 15 seconds.

What the system does: The system marks the 400 verified rows as verified, marks the 80 timed-out rows as pending_verification, and schedules a retry for batch 5 in 60 seconds.

Step 4: Retry. The retry fires. The third-party API returns results for 75 of the 80 rows. 5 rows return a permanent "unverifiable" status.

Step 5: Final state. The database now contains:

• 475 contacts imported and verified
• 5 contacts marked as "unverifiable" (imported but flagged)
• 20 rows rejected during validation (not imported)

What the user sees: A summary screen:

Import complete: 475 contacts imported
5 contacts imported with warnings (unverifiable email)
20 rows skipped -- download error report

The error report CSV contains:

• Row number
• Original data
• Reason for skip ("missing field: last_name" or "duplicate of contact ID 4821")

Why this matters

Without explicit criteria for the timeout (step 3) and the partial-success summary (step 5), an engineer might reasonably implement any of these behaviors: fail the entire batch if any row times out, silently drop the timed-out rows, retry indefinitely, or show a generic "import failed" message. Each of those would be a defensible engineering choice and a bad user experience. The acceptance criteria eliminate that ambiguity before code is written.

Notice also that this flow does not involve irreversible effects -- contacts are written to your own database, so they can be deleted if the import is rolled back. If the flow instead sent a welcome email to each imported contact, the criteria for step 3 would need to change fundamentally: you could not retry the entire batch without risking duplicate emails to the 400 contacts already verified. The irreversible-effect flag from the mapping step is what tells you this distinction matters.

Comparing realistic approaches to edge-case criteria

Most teams should use the checklist approach for simple single-service stories and the coverage matrix for anything that crosses a system boundary or has irreversible effects. The coverage matrix pays for itself the first time it prevents a "what should happen here?" question during code review -- or worse, during an incident.

Failure symptoms, corrective actions, and escalation points

Teams that use delivery and code-quality examination tools can track how often edge-case-related defects appear in production and use that signal to decide which story types need the full coverage matrix.

Explicit limitations

• This guide covers application-level acceptance criteria, not infrastructure resilience. Circuit breakers, retry policies at the HTTP client level, and database failover are engineering concerns that should be documented elsewhere. The criteria in this guide tell engineers what behavior the user should see, not how to implement the retry.
• The coverage matrix does not replace integration testing. It identifies what needs testing. The tests themselves still need to be written, and the failure conditions need to be reproducible in a test environment -- which is not always possible for third-party API failures without mocks.
• Edge-case criteria add planning time. For a simple CRUD feature with no external dependencies, a full coverage matrix is overhead. Use judgment about which stories need this treatment. The heuristic: if the feature crosses a system boundary or triggers an irreversible effect, use the matrix.
• Unknown outcomes are genuinely hard. When a timeout occurs and you cannot determine whether the remote system processed the request, no amount of criteria writing resolves the fundamental ambiguity. The criteria can specify what the user sees and what reconciliation runs, but the underlying distributed-systems problem remains. Be honest about this in your criteria rather than pretending you can guarantee a clean state.
• Compound failures multiply uncertainty. The compound-failure section above provides a heuristic, but real incidents will combine edge cases in ways this guide does not enumerate. The goal is to build the team's reflex for asking "what if two of these happen at once?" -- not to list every combination.
• Criteria do not prevent all production surprises. Failures combine in ways you did not anticipate. The goal is to cover the common cases explicitly and build the team's habit of thinking about failure, not to achieve exhaustive coverage.

Ongoing review routine

After each production incident involving this feature:

1. Check whether the failure that occurred is represented in the coverage matrix.
2. If not, add it. Write the criterion retroactively and ensure a test exists.
3. If the failure is in the matrix but the recovery action did not work as specified, update the recovery action.
4. If the failure was a compound case (two edge-case types at once), add a compound row to the matrix to capture the interaction.

Quarterly (or per release cycle):

1. Pull the list of support tickets and incidents related to error states, stuck records, or unexpected behavior.
2. Group them by edge-case type (timeout, duplicate, stale data, auth change, partial success, permanent rejection, compound).
3. If one category dominates, review the criteria-writing process for that type. Are PMs skipping it? Are engineers not testing it?
4. Share the grouped list in a retrospective. The categories themselves are educational -- they build the team's vocabulary for talking about failure.

When onboarding a new PM or engineer: Walk them through one completed coverage matrix for a shipped feature. Explain each row. Ask them to fill out a matrix for a new story and review it together. This is faster and more effective than a document describing the process in the abstract.

Next-action checklist

• Pick one in-progress story that crosses a system boundary.
• List every external dependency that story touches.
• For each dependency, write one timeout criterion and one permanent-failure criterion.
• For every user-initiated action in the story, write one duplicate-submission criterion.
• Identify any irreversible effects and flag them in the story.
• For each irreversible effect, check whether the external system offers a status-check endpoint. Note the answer in the story.
• Copy the coverage matrix worksheet into the story and fill at least three rows.
• For any story with both irreversible effects and external timeouts, write at least one compound-failure criterion using the heuristic in this guide.
• Review the filled matrix with your engineering lead before implementation begins.
• After the feature ships, check the first five support tickets. If any relate to an edge case not in the matrix, add it retroactively.
• Schedule a quarterly review of edge-case-related incidents using the ongoing review routine above.

The discipline is not in the format. It is in the habit of asking "what happens when this fails?" for every system boundary, every user action, and every batch -- before the code is written, not after the incident.