Salesforce Outage Recovery After Incident 20004433: What Fails Silently Once the Status Page Turns Green

TL;DR

• Incident 20004433 disrupted coreService across all regions from 07:50 UTC on 16 September 2026. It was declared resolved at 18:59 UTC with 15:26 UTC given as the recovery time, a 7 hour 36 minute impact window. The final impacted count was 912 of the 3243 instances in the fleet, revised down from 1036 during the incident.
• The failure was in the login path, so authentication degraded before anything else. Integration users hit it first and hardest.
• When an incident like this clears, the Lightning UI is the first surface back and tells you almost nothing. Everything asynchronous recovers later, partially, or not at all.
• Salesforce does not backfill a missed scheduled job. A CronTrigger whose PreviousFireTime predates the outage window simply did not run, and nothing will tell you. Salesforce reported exactly this on its own status page at 14:00 UTC, for customers who by then could reach the service again.
• Batch Apex can report Completed having processed a fraction of its chunks. Half a nightly reconciliation is worse than none because it looks finished.
• Platform event and Change Data Capture replay windows are measured in hours. Whatever you miss before retention closes is gone.
• Snapshot your schedule inventory while the org is still degraded. After recovery there is nothing left to diff against.
• Five checks here have no CLI route. Knowing which ones saves you hunting for a command that was never built.

What You'll Learn

• What Salesforce published about incident 20004433, and what the failure mode tells you about where to look first
• The one snapshot to capture before recovery, and the diff that makes it worth having
• A CLI query for each class of silent async failure: batch jobs, queueable chains, scheduled jobs, Flow interviews, event subscribers, bulk loads
• How to tell a genuinely expired refresh token from an outage casualty, and why re-authenticating early hides the answer
• Which five checks have no API surface at all

The Problem

An incident clears. Someone loads a record page, it renders, and the org gets declared recovered. That conclusion comes from the only surface a person can see, which is also the surface with the least dependent work behind it.

The expensive damage is asynchronous, and asynchronous failure is quiet by design. A batch job that aborted mid-run does not resume when the platform comes back. A scheduled job that should have fired at 02:00 does not fire at 02:15 instead. A Flow interview that failed at an element is parked in a queue with no owner and no alert. An outbound message that exhausted its retry window is discarded permanently. None of these produce a red banner. Most produce nothing at all.

The gap widens when the outage window overlaps your integration and batch slot. For orgs whose users sit in one timezone and whose nightly processing runs in another, a mid-morning incident somewhere else is your overnight window, and the outage does its real damage while nobody is watching a screen.

There is a second problem that bites later. After recovery you cannot reconstruct what should have happened. CronTrigger holds the next fire time and the last one, never the ones that were skipped. If you did not capture the schedule before recovery, you have no list to reconcile against and no way to build one.

Common questions this article answers:

• What actually happened during the Salesforce outage on 16 September 2026?
• How do I tell whether a scheduled Apex job ran during an outage or was skipped entirely?
• How do I find batch jobs that reported Completed but only processed part of their scope?
• Which post-outage checks can I run from the sf CLI, and which force me into Setup?

Quick Answer

Treat recovery as reconciliation rather than a smoke test. Before the incident clears, export your schedule inventory with sf data query -q "SELECT Id, CronJobDetail.Name, State, PreviousFireTime, NextFireTime, TimesTriggered FROM CronTrigger" to CSV. After the all-clear, export it again and diff the two. Any job whose PreviousFireTime still predates the outage window missed its run, and Salesforce will not backfill it. Then sweep AsyncApexJob for Failed and Aborted jobs and, more importantly, for jobs where JobItemsProcessed != TotalJobItems, which is the partial completion that reads as success everywhere. Query FlowInterview and FlowInterviewLogEntry for paused and failed interviews, EventBusSubscriber for the gap between Position and Tip, and the Bulk API 2.0 ingest endpoint for loads that completed carrying failed rows. Check SetupAuditTrail last, to find out what people changed under pressure while the org was down. Five things have no CLI route: scheduled data export history, the outbound message delivery queue, bulk resume of Flow interviews, Email Log Files, and org-wide sharing recalculation.

What Salesforce published about incident 20004433

Salesforce logged this as a major service disruption against the coreService key on its public status page, flagged as affecting all regions, with impact starting at 07:50 UTC on 16 September 2026. The record is Trust incident 20004433, and everything in this section comes from it.

The public update trail matters because it tells you what kind of failure you are recovering from.

Time (UTC)	Trust update
08:45	Multiple instances across all regions. Severe delays, intermittent errors, inability to access some services. Help portal case creation also affected
09:10	Requests stalling while waiting on a response from an internal login service, using up available server resources. Rolling restart attempted on one impacted instance
09:57	An external dependency failure impacting the legacy login server. API endpoint blocked as a mitigation. Third-party infrastructure provider confirmed no issues on their side
10:01	Restarts abandoned as a path to remediation
10:18	A core system component experienced increased load, limiting its capacity to process requests
10:31	Fix under test on a test instance
10:56	Fix validated on the test instance, rolling out fleetwide
11:19	Fleetwide rollout in progress, customers starting to see service return to normal. A code-level permanent fix still in progress
11:41	Rollout progressing region by region with no confirmed completion time. GovCloud confirmed out of impact
12:39	Most instances recovering, but some still erroring until Salesforce completes a manual restart on that instance. No confirmed resolution time
13:13	Rollout found not to have fully completed on a number of instances, so the fix is being reapplied. Some users need to clear cached data or restart their session
14:00	Reports of scheduled jobs not running for customers who could now reach the service, under investigation. Impact radius described as narrower than first understood
15:26	Service restored. Confirmed later as the recovery time
15:39	Remaining impact narrowed to a subset of Hyperforce instances. First-party environments not impacted
15:56	Impact radius formally decreased. Instances that had not experienced impact removed from the incident
18:59	Incident declared resolved after extended validation, with a full investigation to follow

Salesforce posted routine progress notes between these, roughly every 30 minutes, each one reporting the rollout still moving region by region with services returning to normal in the regions already finished.

Salesforce declared the incident resolved at 18:59 UTC, naming 15:26 UTC as the recovery time. That puts the impact window at 7 hours and 36 minutes, most of it spent on a fix that had already been validated but kept failing to land cleanly across the fleet.

The final impacted count was 912 instances, not the 1036 listed while the incident was running. At 15:56 UTC Salesforce formally decreased the impact radius and removed instances that turned out never to have been affected. Anyone who sized their response off the live figure was working from a number that later moved against them in both directions.

The rootCause, actionPlan and pathToResolution fields were still null at resolution. Salesforce committed to a full investigation covering the technical trigger, the underlying cause and preventive action, so if you need a root cause for an internal incident report, ask your account team rather than waiting for those fields to fill in.

Two details from that trail shape the recovery work. The failure sat in the login path, which means authentication and session handling degraded ahead of everything else, so integration users and scheduled work hit it before any human noticed a slow page. And the platform was shedding load to survive, which is exactly why retrying into it deepened the problem instead of riding it out.

Set up once per shell

Every command below assumes two variables so you can paste them as written.

# the org alias you want to work against
export ORG=my-prod-org

# the start of the impact window, in UTC
export SINCE=2026-09-16T07:50:00Z

Add --json to any sf command for machine-readable output, or --result-format csv --output-file out.csv on sf data query when you need the result as evidence for the incident record. Substitute your own org's API version wherever a REST path appears; the examples use v67.0.

Phase 0: what to do while you are still down

Declare a change freeze, and say so explicitly

No deploys, no config changes, no re-authentication attempts until the incident is marked resolved. Changes made blind during an outage are the usual cause of the follow-on incident, and they contaminate your own analysis afterwards because you can no longer separate platform behaviour from your own edits.

Stop the retry storm

Pause middleware retry loops and schedulers pointed at the affected org. The 09:10 UTC update named the mechanism: requests stalling on the login service were consuming the available server resources. Retrying into that consumed more of them, and guaranteed a thundering herd the moment authentication recovered. A polite integration recovers faster than an aggressive one.

Capture the schedule inventory before recovery

This is the highest value thing you can do during an outage, and the window for it closes when the incident clears.

sf data query -o $ORG --result-format csv --output-file cron-before.csv \
  -q "SELECT Id, CronJobDetail.Name, CronJobDetail.JobType, State,
             PreviousFireTime, NextFireTime, TimesTriggered
      FROM CronTrigger ORDER BY NextFireTime"

If the org is degraded rather than fully unreachable, this usually still completes. It becomes your reconciliation list, and without it the diff in Phase 2 has nothing to compare against.

Check your escalation route works

Trust confirmed at 08:45 UTC that Help portal case creation was itself affected. The web route was closed for the duration, so anyone whose escalation plan begins with "raise a case" had no plan. Your Premier or Signature TAM contact and phone support were the routes that still worked.

Leave expired-looking tokens alone

A refresh that fails with an upstream timeout during a login-path incident is an outage casualty, not a dead token. Re-authenticating now overwrites the evidence and proves nothing, because a successful new auth only tells you the platform came back. Wait for the all-clear, then re-test. A failure that persists afterwards is a real credential problem and you can treat it as one.

Phase 1: the first fifteen minutes after the Salesforce all-clear

Verify each instance individually

Recovery is as uneven as impact was, and a thousand-odd instances recovering fleetwide do not recover together. This incident proved the point twice over: some instances needed a manual restart before they came back, and at 13:13 UTC Salesforce reported that the rollout had not fully completed on a number of instances and was being reapplied. Check the instance your org actually lives on rather than reading a summary feed, and do not assume the bulk list is in step with the per-instance endpoint during an active incident.

for I in NA123 AP456; do
  curl -s "https://api.status.salesforce.com/v1/instances/$I/status" \
    | python3 -c "import json,sys;d=json.load(sys.stdin);print(d['key'],d['status'])"
done

Not knowing your org's instance key is itself a finding. Record it for every org, so the next incident has no blind spot:

sf data query -o $ORG \
  -q "SELECT Id, Name, InstanceName, IsSandbox FROM Organization LIMIT 1"

Confirm recovery is real rather than just non-erroring

Time the round trips and compare them against a normal day. A platform still shedding load will answer slowly for another hour while returning no errors at all. Salesforce also noted that some users would need to clear cached data or restart their session before they saw full recovery, so a healthy API response does not always mean a healthy browser session.

time sf org display -o $ORG --json
time sf data query -o $ORG -q "SELECT Id, InstanceName FROM Organization LIMIT 1"
sf org list limits -o $ORG

Review failed logins across the window

Given where this failure sat, this check earns more attention than usual. Integration users matter more than people here, because a locked service account presents as an interface that runs cleanly and returns nothing.

sf data query -o $ORG -q "SELECT UserId, LoginTime, Status, LoginType,
       SourceIp, Application, ApiType
FROM LoginHistory
WHERE LoginTime >= $SINCE AND Status != 'Success'
ORDER BY LoginTime DESC LIMIT 500"

The Setup equivalent is Setup > Identity > Login History.

Check API quota headroom before reopening integrations

A retry storm burns daily API calls whether or not the calls succeeded. Confirm you have room before unpausing anything.

sf org list limits -o $ORG | grep -i -E "DailyApi|ConcurrentAsync"

Phase 2: the full post-outage sweep

This is where the real damage turns up. Work it in order, because the categories are roughly ordered by how quietly they fail.

Asynchronous Apex: failed, aborted and partially completed batch jobs

Start with the shape of the damage so you know where to spend attention.

sf data query -o $ORG -q "SELECT Status, JobType, COUNT(Id) jobs
FROM AsyncApexJob WHERE CreatedDate >= $SINCE
GROUP BY Status, JobType"

Then pull the failures with their error text. ExtendedStatus carries the actual reason, and aborted jobs never resume on their own.

sf data query -o $ORG -q "SELECT Id, ApexClass.Name, JobType, Status,
       ExtendedStatus, NumberOfErrors, JobItemsProcessed, TotalJobItems,
       CreatedDate, CompletedDate
FROM AsyncApexJob
WHERE CreatedDate >= $SINCE AND Status IN ('Failed','Aborted')
ORDER BY CreatedDate DESC LIMIT 200"

The dangerous case is the one missing from that list. A batch job can finish with status Completed having processed only some of its chunks, and it reads as success in every summary view you have.

sf data query -o $ORG -q "SELECT Id, ApexClass.Name, Status,
       JobItemsProcessed, TotalJobItems, NumberOfErrors
FROM AsyncApexJob
WHERE CreatedDate >= $SINCE AND JobType = 'BatchApex'
  AND (NumberOfErrors > 0 OR JobItemsProcessed != TotalJobItems)"

Anything still sitting in Processing hours after recovery is hung rather than slow:

sf data query -o $ORG -q "SELECT Id, ApexClass.Name, JobType, Status, CreatedDate
FROM AsyncApexJob
WHERE Status IN ('Queued','Preparing','Processing','Holding')
ORDER BY CreatedDate"

Held batches all release at once when the platform recovers, which can turn your recovery into a self-inflicted second incident. FlexQueueItem is a standard object so the queue is queryable, though reordering it still needs Apex or the Setup page.

sf data query -o $ORG -q "SELECT Id, AsyncApexJobId, JobPosition, JobType
FROM FlexQueueItem ORDER BY JobPosition"

# abort a held job rather than let it release into the surge
echo "System.abortJob('707xxxxxxxxxxxxxxx');" > abort.apex
sf apex run -f abort.apex -o $ORG

Queueable chains deserve their own look. A Queueable that fails partway along a chain stops every downstream job, so it surfaces as one failure rather than as the ten jobs that never ran.

sf data query -o $ORG -q "SELECT Id, ParentJobId, Status, ApexClass.Name, CreatedDate
FROM AsyncApexJob
WHERE JobType = 'Queueable' AND CreatedDate >= $SINCE
ORDER BY CreatedDate"

One more worth knowing: unhandled Apex exceptions email the last developer to modify the class by default, not an admin distribution list. Check who is actually receiving them.

sf data query -t -o $ORG -q "SELECT Id, UserId, Email FROM ApexEmailNotification"

Scheduled jobs that did not run, where the real loss usually is

Diff against the snapshot you took in Phase 0.

sf data query -o $ORG --result-format csv --output-file cron-after.csv \
  -q "SELECT Id, CronJobDetail.Name, CronJobDetail.JobType, State,
             PreviousFireTime, NextFireTime, TimesTriggered
      FROM CronTrigger ORDER BY NextFireTime"

diff cron-before.csv cron-after.csv

A PreviousFireTime that still predates the outage window means the job did not run, and Salesforce does not backfill a missed fire time. The scheduler moves on to the next occurrence. Whatever that job was responsible for did not happen, and the only trace is a timestamp that failed to advance.

Salesforce confirmed this pattern itself during the incident. The 14:00 UTC update reported that customers who could reach the service again were seeing scheduled jobs not running as expected, and that it was investigating. That is the failure this section describes, surfacing on the vendor's own status page roughly six hours after impact began and while many orgs would have considered themselves recovered.

Separately, find schedules that are no longer waiting to run at all:

sf data query -o $ORG -q "SELECT Id, CronJobDetail.Name, State, NextFireTime
FROM CronTrigger WHERE State != 'WAITING'"

A job in ERROR, PAUSED or DELETED state is dead until a human reschedules it, and nothing monitors that for you.

CronJobDetail.JobType is a coded value and the coding has shifted between releases, so read the mapping out of your own org rather than trusting a table you found somewhere. Scheduled Apex, scheduled Flows, dashboard refreshes and reporting snapshots all live in the same object under different codes.

sf data query -o $ORG -q "SELECT CronJobDetail.JobType, COUNT(Id) n
FROM CronTrigger GROUP BY CronJobDetail.JobType"

sf data query -o $ORG -q "SELECT Id, Name, JobType FROM CronJobDetail ORDER BY JobType"

Scheduled-triggered Flows are worth isolating because they fail differently from Apex. A failed scheduled Flow emails the last person to modify it and stops. There is no retry and no queue entry.

sf data query -t -o $ORG -q "SELECT Id, MasterLabel, ProcessType, Status,
       TriggerType FROM FlowDefinitionView
WHERE TriggerType = 'Scheduled'"

Cross-reference those definitions against the CronTrigger rows to confirm each one fired. Dashboard refreshes and reporting snapshots are also CronTrigger backed and also fail silently, and a reporting snapshot that skipped a night leaves a permanent hole in trend data that cannot be filled in later.

Re-run what should have fired, staggered rather than all at once:

echo "Database.executeBatch(new MyNightlyBatch(), 200);" > rerun.apex
sf apex run -f rerun.apex -o $ORG

Paused and failed Flow interviews

This queue exists in every org and gets opened in almost none of them. You do not need the Setup page to find its contents, only to act on them in bulk.

sf data query -o $ORG -q "SELECT Id, InterviewLabel, CurrentElement,
       PauseLabel, InterviewStatus, CreatedDate, CreatedBy.Name
FROM FlowInterview
WHERE CreatedDate >= $SINCE
ORDER BY CreatedDate DESC"

The failure detail lives in the interview log, which is richer than the Setup summary because it names the element that threw.

sf data query -o $ORG -q "SELECT Id, FlowInterviewLogId, ElementApiName,
       ElementType, ErrorMessage, CreatedDate
FROM FlowInterviewLogEntry
WHERE CreatedDate >= $SINCE
ORDER BY CreatedDate DESC LIMIT 200"

If a field name is rejected, run sf sobject describe -s FlowInterviewLogEntry -o $ORG rather than guessing. These objects have gained fields across recent releases.

Two related checks have no query of their own and still matter. Async paths on record-triggered Flows run in a separate transaction and fail independently of the record save, so the record looks correct while the follow-on work never happened. And approval submissions interrupted mid-outage leave records locked with no pending approver, which users will not report because they assume it is sitting with someone else.

sf data query -o $ORG -q "SELECT Id, TargetObjectId, Status, CreatedDate
FROM ProcessInstance
WHERE Status = 'Pending' AND CreatedDate >= $SINCE"

Platform events and Change Data Capture

EventBusSubscriber is a standard object rather than a Tooling one, and the gap between Position and Tip is your backlog.

sf data query -o $ORG -q "SELECT Name, Type, Position, Tip, Retries,
       LastError, Status, ExternalId
FROM EventBusSubscriber
ORDER BY Retries DESC"

Replay is the one genuinely time-boxed item in this runbook. High-volume platform events and Change Data Capture are retained for roughly 72 hours, standard-volume events for considerably less. Whatever you have not replayed when the window closes is unrecoverable. The sf CLI has no subscribe command, so replay means a Pub/Sub API client or a CometD subscriber holding your stored replay ID.

A collapse in publish volume across the window tells you the publisher was blocked rather than the subscriber falling behind, which points the investigation somewhere completely different:

sf data query -o $ORG -q "SELECT Name, StartDate, EndDate, Value
FROM PlatformEventUsageMetric
WHERE StartDate >= $SINCE"

Integrations, inbound and outbound

Overnight bulk loads are the classic casualty. A Bulk API 2.0 job can report as completed while carrying a high failed-record count, so check the record counts rather than the job state.

sf api request rest "/services/data/v67.0/jobs/ingest" -o $ORG

# failed rows for a specific job
sf data bulk results --job-id <jobId> -o $ORG

Outbound messages retry for up to 24 hours and are then discarded permanently. The definitions are queryable through Tooling, but the runtime delivery queue is not exposed to any API, so the pending and failed counts have to be read in Setup.

sf data query -t -o $ORG -q "SELECT Id, Name, EndpointUrl, ApiVersion
FROM WorkflowOutboundMessage"

Then go hunting for duplicates. Any interface without an idempotency key or an external-ID upsert will have created doubles when it retried into a partially-responsive platform, and a retried POST against an intermittent error is exactly that situation.

sf data query -o $ORG -q "SELECT External_Id__c, COUNT(Id) n
FROM My_Object__c
WHERE CreatedDate >= $SINCE
GROUP BY External_Id__c HAVING COUNT(Id) > 1"

Two inbound channels behave differently and both need attention. Web-to-Case and Web-to-Lead fail silently at the browser with no queue and no retry, so submissions made during the window are lost and the only surviving record is your web server log. Email-to-Case is the opposite, because the sending mail server retries, so cases arrive late and out of order rather than not at all.

sf data query -o $ORG -q "SELECT Id, Subject, MessageDate, Status, Incoming
FROM EmailMessage
WHERE MessageDate >= $SINCE AND Incoming = true
ORDER BY MessageDate"

Finally, review Apex callout failures. A callout that timed out mid-transaction may have committed on the remote system while rolling back locally, which is the worst class of inconsistency available because both systems then believe they are correct.

sf apex list log -o $ORG
sf apex get log --number 10 -o $ORG | grep -i "CALLOUT\|System.CalloutException"

Data integrity

Record counts give you a fast org-wide baseline, which is the only check that reliably catches silent partial loss:

sf org list sobject record-counts -o $ORG --json > counts-after.json

Use SystemModstamp rather than LastModifiedDate when scoping to the window. It is indexed, and it captures system-driven updates that LastModifiedDate can miss.

sf data query -o $ORG -q "SELECT COUNT(Id) FROM Case
WHERE SystemModstamp >= $SINCE"

Include deleted rows when you suspect rolled-back transactions. --all-rows surfaces recycle bin records that a failed transaction left behind.

sf data query -o $ORG --all-rows -q "SELECT Id, IsDeleted, CreatedDate
FROM Case WHERE CreatedDate >= $SINCE AND IsDeleted = true"

Change control and audit

Run this one even if you are confident nobody touched anything, because it is how you find out that somebody did.

sf data query -o $ORG -q "SELECT Action, Section, CreatedBy.Name,
       CreatedDate, Display
FROM SetupAuditTrail
WHERE CreatedDate >= $SINCE
ORDER BY CreatedDate DESC"

Undocumented emergency edits are a frequent cause of the next incident, and the audit trail is the only place they are recorded. Check deployments in the same pass:

sf project deploy report -o $ORG
sf data query -t -o $ORG -q "SELECT Id, Status, StartDate, CompletedDate,
       NumberComponentErrors, CreatedBy.Name
FROM DeployRequest
WHERE CreatedDate >= $SINCE ORDER BY CreatedDate DESC"

Trace flags expire on a timer, so any diagnostic logging you were relying on has probably lapsed:

sf data query -t -o $ORG -q "SELECT Id, TracedEntityId, ExpirationDate,
       LogType FROM TraceFlag ORDER BY ExpirationDate DESC"

Then lift the change freeze as explicitly as you declared it. An implicit end leaves half the team frozen and the other half deploying.

The five checks with no CLI route

Knowing where the API stops saves an hour of searching for a command that was never built.

Check	Where it lives	Why it matters
Scheduled data export history	Setup > Data Export	Export history is not exposed to the API, and a missed export is often a compliance artefact
Outbound message delivery queue	Setup > Environments > Monitoring > Outbound Messages	Definitions are queryable through Tooling; the pending and failed runtime queue is not
Bulk resume of Flow interviews	Setup > Process Automation > Paused and Failed Flow Interviews	Finding them is SOQL, acting on them in bulk is UI only
Email Log Files	Setup > Email Log Files	Request and download only, with a delay before the file is available
Org-wide sharing recalculation	Setup > Sharing Settings > Recalculate	Cannot be triggered from Apex or the API, and matters if recalculation was interrupted

You can at least jump straight to the right page from the terminal:

sf org open -o $ORG -p /lightning/setup/PausedFlowInterviews/home

Phase 3: the day after the Salesforce outage

Request the official root cause analysis through your account team rather than waiting for it to appear on the public feed. The Trust record for 20004433 was declared resolved at 18:59 UTC with all three of those fields still empty, alongside a commitment to investigate the technical trigger and underlying cause. Resolved incidents also drop off the public feed eventually. Archive it while it is still there:

curl -s "https://api.status.salesforce.com/v1/incidents/20004433" \
  > incident-20004433-$(date +%Y%m%d).json

Then do the part that pays for the outage. Every failure you discovered by querying rather than by receiving an alert is a missing alert, and silent scheduled-job failure is almost always top of that list. The retry design gaps this incident exposed are worth fixing while the memory is fresh: exponential backoff with jitter, a dead-letter queue, idempotency keys or external-ID upserts on every inbound interface, and a circuit breaker so your integrations back off from a platform that is already shedding load. An outage with a public incident number attached is the easiest business case you will get for that work.

Frequently Asked Questions

Q: What caused the Salesforce outage on 16 September 2026?

A: Salesforce has not published one. The incident was declared resolved at 18:59 UTC on 16 September 2026, with 15:26 UTC given as the recovery time and a 7 hour 36 minute impact window, and the rootCause, actionPlan and pathToResolution fields on the Trust record were all still empty at that point. Salesforce committed to a full investigation into the technical trigger and underlying cause. What the update trail describes is requests stalling while waiting on an internal login service and consuming available server resources, attributed at 09:57 UTC to an external dependency failure affecting the legacy login server. Salesforce blocked the relevant API endpoint as mitigation, confirmed with its infrastructure provider that the problem was not theirs, and by 10:18 UTC described a core system component under load beyond its capacity to process requests. A fix was validated on a test instance at 10:56 UTC and rolled out from there, with the 11:19 UTC update reporting service starting to return to normal and later updates describing a rollout that repeatedly failed to land cleanly, needing manual restarts and a reapplication before the remaining impact was narrowed to a subset of Hyperforce instances.

Q: Does Salesforce re-run a scheduled Apex job that was missed during an outage?

A: No. The scheduler advances to the next occurrence and the missed one is not backfilled. The only visible evidence is that PreviousFireTime on the CronTrigger record did not advance past the outage window, which is why capturing the schedule before recovery matters. Without that snapshot you can still spot obviously stale fire times, but you cannot separate a job that was skipped from one that is simply infrequent.

Q: A batch job says Completed. Is that enough?

A: Not on its own. Completed describes the job reaching the end of its execution, not the scope being fully processed. Compare JobItemsProcessed against TotalJobItems and check NumberOfErrors. A job that processed 40 of 120 chunks and stopped cleanly still reports Completed, and a partially-run reconciliation is worse than one that never ran, because it presents as finished and so nobody reruns it.

Q: How long do I have to replay platform events after an outage?

A: Roughly 72 hours for high-volume platform events and Change Data Capture, and considerably less for standard-volume events. This is the only irreversible deadline in the runbook, so if your event backlog is large, start the replay before you finish the rest of the sweep. Standard-volume platform events are also on a retirement path, which shortens the retention picture further for orgs still using them.

Q: Our refresh tokens failed during the outage. Should we re-authenticate immediately?

A: Wait for the all-clear. Given that this incident sat in the login path, a refresh failing with an upstream timeout was almost certainly the incident rather than a dead token, and re-authenticating overwrites the state that would have told you which. Re-test once the incident is resolved. A failure that persists after recovery is a real credential problem and you can treat it as one.

Q: We only have five minutes. What do we run?

A: The CronTrigger diff and the partial-batch query. Between them they cover the two failure modes that are both common and completely silent. Everything else in this runbook either announces itself eventually or shows up in a user complaint.

Key Takeaways

• Incident 20004433 was a login-path failure, which is why authentication, integration users and scheduled work degraded before anyone noticed a slow page.
• The UI recovering tells you almost nothing. It is the first surface back and the one with the least dependent work behind it.
• Missed scheduled jobs are never backfilled, so the diff against a pre-recovery snapshot is the highest-value check available, and it has to be set up before the incident clears.
• Completed is not the same as complete. Compare JobItemsProcessed to TotalJobItems on every batch job in the window.
• Replay windows expire in hours, which makes platform event and CDC backlogs the only genuinely time-boxed item on the list.
• Most of the sweep is CLI-addressable, through SOQL, Tooling SOQL, REST passthrough and anonymous Apex. Exactly five checks are not.
• Every failure you found by querying is a missing alert. That list is the real output of the incident, more than the root cause analysis you will eventually be given.

What's Next?

Recommended Reading:

• Why filtering Salesforce event logs by IP misses most of the incident for reconstructing what happened once you have the logs
• Free event monitoring with EventLogFile for the capture habit that makes post-incident analysis possible at all
• Measure and reduce platform event delivery limits for the subscriber capacity that decides how fast a backlog drains
• Standard-volume platform events retirement for why your replay window may be shorter than you think
• Connect the sf CLI to an org if any of the commands above are new to you

Action Items:

1. Put the Phase 0 CronTrigger export into your incident runbook today, so it is a paste rather than a decision when the next outage starts.
2. Record the instance key for every production org and store it somewhere reachable when the org is not.
3. Add an alert on scheduled jobs whose State is not WAITING, the cheapest monitoring gap on this list to close.
4. Check that your escalation path does not depend on the Help portal, which was unavailable for the duration of this incident.
5. Audit your inbound interfaces for idempotency keys or external-ID upserts, and fix the ones that have neither before they duplicate records for you.

Resources & References

• Salesforce Trust incident 20004433
• Salesforce Trust status API
• AsyncApexJob object reference (Salesforce Developers)
• CronTrigger object reference (Salesforce Developers)
• FlowInterview object reference (Salesforce Developers)
• EventBusSubscriber object reference (Salesforce Developers)
• Change Data Capture Developer Guide (Salesforce Developers)
• Bulk API 2.0 ingest jobs (Salesforce Developers)
• Salesforce CLI command reference