Brave Search

who else is on-call? let's goooo

1 of 76

418

2 of 76

114

Don't worry guys, AI will fix it

reddit.com › r/experienceddevs › why was aws outage so devastating?

r/ExperiencedDevs on Reddit: Why was AWS outage so devastating?

October 21, 2025 -

AWS Global Infrastructure

The AWS Cloud spans 120 Availability Zones within 38 Geographic Regions, with announced plans for 10 more Availability Zones and 3 more AWS Regions in the Kingdom of Saudi Arabia, Chile, and the AWS European Sovereign Cloud.

I thought that for companies like Amazon, Delta, Snapchat, Google and Venmo multi region setup was standard. One of the main premises of cloud services is the resilience to outage of one region or node. And yet, once us-east-1 is down, it's all over.

Was that the fault of AWS or those who used AWS tied to one region?

Edit: from the responses I came to conclusion that I'm gonna have my own resiliency with ~~blackjack and hookers~~ nginx and multiple cloud providers and it probably gonna work better than AWS.

Multi region redundancy is expensive, complicated and probably not worth it, so many companies don't do it. Combined with the fact that us east 1 is the largest and oldest region, when it goes down, a lot of things go down with it

1 of 66

805

2 of 66

322

AWS themselves use us-east-1 to host a lot of services that help the other regions to run. So when those services break it affects the usability in other regions. The extent to which a given company was effected depends on how well they anticipated such a failure.

reddit.com › r/worldnews › worldwide aws outage affects hundreds of millions

r/worldnews on Reddit: Worldwide AWS outage affects hundreds of millions

September 8, 2025 - As our company runs headfirst into putting everything possible into the cloud it begs the questions. Are you willing to suffer a full day outage? We are about to put the last 25% there and these are some pretty serious systems that impact out ability to operate if they are unavailable. Guess where the documentation was to recover from a failure??! Yep AWS.

reddit.com › r › aws

Amazon Web Services (AWS): S3, EC2, SQS, RDS, DynamoDB, IAM, CloudFormation, Route 53, VPC and more

October 27, 2025 - r/aws: News, articles and tools covering Amazon Web Services (AWS), including S3, EC2, SQS, RDS, DynamoDB, IAM, CloudFormation, AWS-CDK, Route 53, CloudFront, Lambda, VPC, Cloudwatch, Glacier and more.

reddit.com › r/technology › massive aws outage takes down snapchat, reddit, alexa, ring and much of the interne

r/technology on Reddit: Massive AWS outage takes down Snapchat, Reddit, Alexa, Ring and much of the interne

October 20, 2025 - The databricks official status update pretty much said that they aren't doing anything and just waiting on aws to fix it: Databricks Status: [Identified] - Starting at 06:55 UTC October 20, 2025, customers may experience degraded performance with Databricks services. Running jobs may not be completed on time. The cloud provider is actively working on a mitigation. ... Its also great that the outage is front-page news so when you tell them its an AWS problem and not us they really believe you.

reddit.com › r/aws › non-tech here, curious on aws outage affecting multiple sites all day

r/aws on Reddit: Non-Tech Here, Curious on AWS Outage Affecting Multiple Sites All Day

October 20, 2025 -

Hi All,

As title suggests, I just popped in as a non-technical non-user aside from knowing that Flickr is down and has been all day long now, and apparently many other large sites, Reddit included.

Anyone here know the real deal and what's what and can explain it to me like I'm 5?

When a website has to handle people (users) going to the website and making use of the services on it that requires computers actually processing those decisions and actions by the users. In the past businesses would do this themselves by having dedicated computers (servers) handle that processing. They would have these computers on-premises and have their own IT handle expansion and maintenance of those servers. Nowadays most of the businesses and websites on the internet no longer have their own computers because of the time and investment needed to manage it and just rent the usage of computers from Amazon, Microsoft, Google, or whatever hosting they choose to use. AWS normally has a 99.999% availability meaning that it normally has like a 5 minute total downtime for the entire year and for the most part they're very consistent about maintaining that availability. When it goes down that means the businesses that rent the usage from Amazon don't have any way to actually process incoming requests to their website and don't have a backup in place (because this usually does not happen) meaning that the websites go down. Huge amounts of the internet including the sites you mentioned use Amazon to host their sites and so when it goes down they also do.

In Flickr terms, imagine AWS as a giant version of Flickr. Not just for photos, but for everything. Apps, websites, and data. Companies use AWS to store their files and run their services, just like you upload photos to Flickr. AWS is built with a lot of backups and redundancies. It keeps copies of things in multiple “regions” which is just different datacenters across the world. Normally, if one has a problem, another can pick up the slack. But sometimes, very rarely, a major issue hits an entire region. When that happens, every app or service that depends on that specific region can go down. And because some AWS tools depend on other AWS tools (even across regions) it can trigger a domino effect. One outage cascades into others, and suddenly you got a big ol' ball of clusterfuck.

reddit.com › r/experienceddevs › aws outage

r/ExperiencedDevs on Reddit: AWS Outage

August 27, 2025 -

This AWS outage reminded me of how reliant many shops are on the platform. Do you think anyone will move towards a different cloud provider or a multi-cloud approach to ensure stability? Or just chalk it up to a black swan event and move on.

We’ll get a Jira ticket to make our service multi region. We’ll respond with the estimated cost. Higher ups will say the current single region is an acceptable risk given the costs. And so it goes.

1313

the simple trick is to be on us-east-2 instead of us-east-1

289

reddit.com › r/selfhosted › aws is down. who's laughing right now?

r/selfhosted on Reddit: AWS is down. Who's laughing right now?

September 19, 2025 -

Love my Immich instance on a $15/month VDS. Still going strong when half the internet is down.

Gonna be “ one of those days “ in work

1 of 69

485

2 of 69

358

Between Cloudflare and the 3 big clouds, the internet has become very centralized.

reddit.com › r/askreddit › how is the global amazon aws outage affecting you today?

r/AskReddit on Reddit: How is the global Amazon AWS outage affecting you today?

August 18, 2025 - Procore is heavily using AWS. ... Same! Except it’s all of our superintendents, plus project managers. I need to get subcontractor invoice and insurance information entered and can’t get past the log in screen. ... We’re going to have to deal with unhappy customers because “global outage” means nothing to people and customers don’t realize that a ton of small companies like ours use AWS.

Find elsewhere

Google Bing Mojeek

reddit.com › r/aws › how tf did aws mess up so bad that the entire us-east-1 region is down, all 6 azs are fucked.

r/aws on Reddit: How TF did AWS mess up so bad that the entire us-east-1 region is down, all 6 AZs are fucked.

October 20, 2025 -

Isn't the point of availability zones to prevent shit like this from happening?

Lmao datadog and slack are down so I learned about our company outage through Reddit

1 of 56

223

2 of 56

135

Dynamo is dead cant even resolve `dynamodb.us-east-1.amazonaws.com` it's affecting IAM which affects every other services.

reddit.com › r/askengineers › why wasn't aws redundant enough to survive the server outage the other day?

r/AskEngineers on Reddit: Why wasn't AWS redundant enough to survive the server outage the other day?

October 22, 2025 -

I've heard a ton about "Well everything's on the cloud, so a server goes down, and there goes the whole internet" which does not really make sense to me on some level. Isn't this stuff multiple-times redundant? Aren't there fallbacks, safeties, etc?

I thought modern networks are de-centralized and redundant. Why wasn't AWS?

The tl;dr is they screwed up one of the single points of failure of an otherwise highly robust and decentralized database system - a DNS record for the DynamoDB cloud database. So even though the database itself didn't go down, most of the applications that use it couldn't actually connect to it because they couldn't find its number in the virtual phonebook. This particular database happens to be a dependency of many, many other cloud systems, including within AWS internally, and when they were unable to use it, they stopped working, thus causing a cascading set of failures.

1 of 20

313

2 of 20

Certain AWS services are internally redundant within a region as a part of the service. A good example are S3 buckets which are spread across multiple data centers to ensure geographical redundancy. If S3 goes down in one data center in a region, there are a minimum of two other copies in other data centers in the same region which can fill the request. However, this does not extend to all services. Other services such as EC2 require the customer to build redundancy into their application and pay for it accordingly. Customers that properly built their services to be redundant across regions were not affected by the Virginia outage.

reddit.com › r/technology › massive aws outage takes down major services – roblox, fortnite, coinbase, snapchat and all supercell games offline

r/technology on Reddit: Massive AWS outage takes down major services – Roblox, Fortnite, Coinbase, Snapchat and all Supercell games offline

September 16, 2025 - Serious question, we now know that things from signal app to Reddit and multiple airlines at their systems down from this issue which is a AWS issue. To my understanding this was one region of AWS, but AWS offers the ability to have failover. I am assuming that fail over capability is so expensive that literally no one but maybe a few, of the wealthiest companies can afford it?

reddit.com › r/sysadmin › [ removed by moderator ]

AWS outage: Proof the internet's original design has been ...

August 28, 2025 - The Internet worked fine throughout the entire outage. Applications failed due to their shitty architecture. ... Yup, I work for a smaller company (subjectively, several million unique monthly app users) and while we're primarily GCP we have a synced up failover copy of our entire kubernetes cluster at amazon. It takes us 5 minutes to manually failover, or there are automatics then it's just down to DNS propagation.. While a cloudflare situation that takes out both AWS and GCP still takes us out, just AWS or just GCP having issues has little to no effect.

reddit.com › r/wellthatsucks › aws outage

r/Wellthatsucks on Reddit: AWS Outage

August 21, 2025 -

If I lose my 436 day streak over this, I'm going to be really pissed. I've tried commenting. I've tried up votes on Posts and Comments. Now I'm trying a new Post.

Well your post went through at least It's spotty, let me see if this comment shows up

1866

Obtaining a Reddit streak seems like the least valuable thing I can care about. Props though.

1309

reddit.com › r/dataengineering › [megathread] aws is on fire

r/dataengineering on Reddit: [Megathread] AWS is on fire

October 21, 2025 -

EDIT EDIT: This is a past event although it looks like there are still errors trickling in. Leaving this up for a week and then potting it.

EDIT: AWS now appears to be largely working.

In terms of possible root cases, as hypothesised by u/tiredITguy42:

So what most likely happened:

DNS entry from DynamoDB API was bad.

Services can't access DynamoDB

It seems AWS is string IAM rules in DynamoDB

Users can't access services as they can't get access to resources resolved.

It seems that systems with main operation in other regions were OK even if some are running stuff in us-east-1 as well. It seems that they maintained access to DynamoDB in their region, so they could resolve access to resources in us-east-1.

These are just pieces I put together, we need to wait for proper postmortem analysis.

As some of you can tell, AWS is currently experiencing outages

In order to keep the subreddit a bit cleaner, post your gripes, stories, theories, memes etc. into here.

We salute all those on call getting shouted at.

I thought it was literally on fire for a sec lol

109

I was shouting at the analysts, my bad

reddit.com › r/seattlewa › when senior engineers leave seattle - amazon web services (aws) outages are the result

r/SeattleWA on Reddit: When senior engineers leave Seattle - Amazon Web Services (AWS) outages are the result

October 23, 2025 -

The Amazon Web Services (AWS) outage on Monday doesn’t seem to have been caused by a cyberattack. But critics are already pointing to the fact that Amazon has laid off at least 27,000 employees, including – surprise – senior engineers, since 2022.

https://aws.amazon.com/message/101925/ For anyone curious about the actual cause, and not complaints about laying people off.

145

I'll save you a click. The whole article is pure speculation and is based on zero data.

reddit.com › r/cybersecurity › what to know about the amazon web services outage

r/cybersecurity on Reddit: What to know about the Amazon Web Services outage

September 7, 2025 - This is in response to your article ... cloud outage that exposed the internet’s vulnerable backbone." While the article is technically correct, the title is not and is misleading. As a retired networking and cybersec pro, I need to point out that the Internet backbone, which you can think of the freeway that connects you to the services that are you using on the Internet, was not affected and was still operating normally throughout the AWS ...

reddit.com › r/stocks › so amazon caused massive world-wide outage for hours disrupting thousands of buisnesses and costing probably bilions but stock dosen't care?

r/stocks on Reddit: So amazon caused massive world-wide outage for hours disrupting thousands of buisnesses and costing probably bilions but stock dosen't care?

October 21, 2025 -

It's so weird, the main reason people use AWS is for safety and stability, this fails and fails massively but somehow it dosen't move stock even a little bit?

What is going on with this market? Does CEO needs to commit war crimes on pandas or smth for stock to go down (of course if this CEO is Elon, then Tesla would go to new ATH)

Edit: Ok, I clearly don't understate the stock or psychology. Crowdstrike created massive outage - stock get massive hit, pepole say that they are evil, their monopoly is bad and will be broken, everybody panics and company will probably go under. Amazon creates massive outage - yeah, bullish af, they are evil, their monopoly is great and we hope it continues, everybody cheers that so many companies can be affected.

it reminds people of the importance of AWS, so it's bullish

1 of 146

2146

2 of 146

909

Just proves how many companies are paying Amazon on a daily basis. The outage is very bullish in a weird way.

reddit.com › r › pcmasterrace › comments › 1pjg749 › yall_ever_wonder_why_now_we_get_these_worldwide

Y'all ever wonder why now we get these worldwide AWS ...

We cannot provide a description for this page right now

reddit.com › r/sysadmin › amazon explains the cause behind tuesday’s massive aws outage

r/sysadmin on Reddit: Amazon explains the cause behind Tuesday’s massive AWS outage

October 19, 2021 -

Short summary: https://www.bleepingcomputer.com/news/technology/amazon-explains-the-cause-behind-tuesday-s-massive-aws-outage/

Full summary: https://aws.amazon.com/message/12721/

There... does not appear to actually be a root cause posted in here. At 7:30 AM PST, an automated activity to scale capacity of one of the AWS services hosted in the main AWS network triggered an unexpected behavior from a large number of clients inside the internal network. This is not a root cause unless the "unexpected behavior" is explained. I feel like Amazon has been more thorough and transparent in similar public post-mortems in the past. This feels pretty hand-wavey by comparison.

149