During a 2017 S3 disruption, AWS’s dashboard could not update for two hours

On February 28, 2017, an authorized Amazon S3 team member entered an input incorrectly while investigating a slowdown in S3’s billing system in Northern Virginia. A larger set of servers was removed than intended, affecting core S3 subsystems. S3 began serving some requests again at 12:26 PM PST and was operating normally by 1:54 PM PST, after the 9:37 AM PST event began.

There was another problem: until 11:37 AM PST, AWS could not update individual service statuses on its Service Health Dashboard. The dashboard administration console depended on S3, the service having the disruption. AWS used its X feed and a dashboard banner meanwhile, then changed the console to run across multiple AWS Regions. For any website owner, the lesson is simple: keep your incident page and communications path separate from the systems they describe.

Each fact is checked against the primary sources listed. Spotted an error? Email the Status Is UP team. Status Is UP helps owners improve their sites; it is not an uptime monitor. For live outage status, see Status Is Down.