2024 CrowdStrike-related IT outages

2024 CrowdStrike incident

Multiple blue screens of death caused by a faulty software update on airport luggage conveyor belts at LaGuardia Airport, New York City
Date	19 July 2024 (2024-07-19)
Location	Worldwide
Type	IT outage, computer crash
Cause	Faulty CrowdStrike software update
Outcome	~8.5 million Microsoft Windows operating systems crash worldwide, causing global disruption of critical services

On 19 July 2024, American cybersecurity company CrowdStrike distributed a faulty update to its security software that caused widespread problems with computers running Microsoft Windows. As a result, roughly 8.5 million systems crashed and were unable to properly restart in what has been called the largest outage in the history of information technology and "historic in scale".

The outage disrupted daily life, businesses, and governments around the world. Many industries were affected—airlines, airports, banks, hotels, hospitals, manufacturing, stock markets, broadcasting, gas stations, retail stores, and more—as were governmental services, such as emergency services and websites. The worldwide financial damage has been estimated to be at least US$10 billion.

Within hours, the error was discovered and a fix was released, but because affected computers had to be fixed manually, outages continued to linger on many services.

Background

CrowdStrike produces a suite of security software products for businesses, designed to protect computers from cyberattacks. The Falcon Sensor product, CrowdStrike's vulnerability scanner, installs an endpoint sensor at the operating system kernel level on individual computers to detect and prevent threats. Patches are routinely distributed by CrowdStrike to its clients to enable their computers to address new threats.

Outage

On 19 July at 04:09 UTC, CrowdStrike distributed a faulty configuration update for its Falcon sensor software running on Windows PCs and servers. A modification to a configuration file which was responsible for screening named pipes, Channel File 291, caused an out-of-bounds memory read in the Windows sensor client that resulted in an invalid page fault. The update caused machines to either enter into a bootloop or boot into recovery mode.

Almost immediately, Windows virtual machines on the Microsoft Azure cloud platform began rebooting and crashing, and at 06:48 UTC, Google Compute Engine also reported the problem. The problem affected systems running Windows 10 and Windows 11 running the CrowdStrike Falcon software. Most personal Windows PCs were unaffected, as CrowdStrike's software is primarily used by organizations. The CrowdStrike software did not provide a way for subscribers to delay the installation of its content files. Computers running macOS and Linux were unaffected, as the problematic content file was only for Windows, but similar problems had affected Linux distributions of CrowdStrike software in April 2024.

CrowdStrike reverted the content update at 05:27 UTC, and devices that booted after the revert were not affected.

At 07:15 UTC, Google said that the CrowdStrike update was at fault. Within hours, CrowdStrike CEO George Kurtz confirmed that CrowdStrike's faulty kernel configuration file update had caused the problem. At 09:45 UTC, Kurtz confirmed that the fix was deployed and that the problem was not the result of a cyberattack.

The day before the faulty update, the Azure platform had an outage that blocked some companies' access to their storage and to Microsoft 365 applications in Azure's Central United States region. Microsoft said the 18 July incident was unrelated to the CrowdStrike problem, but that the two incidents compounded problems for these customers.

Remedy

Affected machines could be restored by booting into safe mode or the Windows Recovery Environment and deleting any .sys file beginning with C-00000291- and with timestamp 0409 UTC in the %windir%\System32\drivers\CrowdStrike\ directory. As this process needed to be done locally on each individual machine, it was "expected to take days" for affected businesses to restore all systems. Technical staff needed to reboot the affected computers individually with manual intervention on each system, which could be a "monumental task".

On devices with Windows' BitLocker disk encryption enabled, which corporations often use to increase security, fixing the problem was exacerbated because a recovery key could be required that was stored on a server that itself had crashed.

Some Microsoft Azure customers discovered that they could fix the problem by rebooting affected virtual machines up to 15 times, ideally while connected to Ethernet. Microsoft has also recommended restoring a backup from before 18 July to fix the issue.

Impact

Outages were experienced worldwide, reflecting the wide use of Microsoft Windows and CrowdStrike software by global corporations in numerous business sectors. At the time of the incident, CrowdStrike said it had more than 24,000 customers, including nearly 60% of Fortune 500 companies and more than half of the Fortune 1000. On 20 July, Microsoft estimated that 8.5 million devices were affected by the update, which it said was less than one percent of all Windows devices.

Widespread outages were immediately reported across multiple countries, with major global disturbances experienced by the general public sweeping from east to west from time zone to time zone. At 04:09 UTC on 19 July, the time when the faulty update was issued, it was the middle of the business day of Oceania and Asia, the early morning hours in Europe, and midnight in much of the Americas.

Some countries were less affected. China, which has striven toward self-sufficiency in IT, saw little impact to key services such as airlines and banks, although foreign businesses and luxury hotels in the country were affected. Russia and Iran—both restricted by international sanctions from using the services of American high-tech companies—reported no disruptions.

A specialist cloud outage insurance business estimated that the top 500 US companies by revenue, excluding Microsoft, had faced near $5.4bn (£4.1bn) in financial losses because of the outage, but only between $540m (£418m) to $1.08bn (£840m) of those losses would be insured.

CrowdStrike liability

Despite the losses companies have suffered, CrowdStrike was said to be only minimally liable for the damage or lost revenue caused. The terms for CrowdStrike's Falcon software limits liability to 'fees paid', so the maximum compensation an affected company could recover if this provision is enforceable are the fees that the company has paid to CrowdStrike.

In the EU, it is possible that CrowdStrike will be held liable under a GDPR regulation related to the impact of security incidents on user data. The regulation is best known in relation to data leaks but also applies to data destruction. It is unclear whether temporary loss of access to data is enough to trigger liability, or whether GDPR applies to all incidents related to security or only unauthorised access.

Air transport

Digital signage at Dulles International Airport displaying a blue screen of death during the incident

Globally, 5,078 air flights, 4.6% of those scheduled that day, were cancelled.

Oceania

Australian airlines Qantas, Virgin Australia, and Jetstar were affected. A Sydney Airport spokesperson said that the outage had affected some operations and that "there may be some delays throughout the evening". Melbourne Airport saw check-in procedures disrupted; officials advised passengers to consult with their airlines. The Adelaide, Brisbane, Canberra, Darwin, Hobart, Launceston, and Perth airports were also affected. In New Zealand, Christchurch Airport was having problems.

Asia

AirAsia passengers queue at Ngurah Rai International Airport, Indonesia, after the airline's check-in system was disrupted.

Hong Kong International Airport experienced delays during check-in, primarily for passengers of the local budget carrier Hong Kong Express, whose staff members used handwritten signs to direct passengers to check-in counters. The Hong Kong Airport Authority activated an emergency response after airline websites and automatic check-in malfunctioned. The booking systems of local airlines Cathay Pacific, Hong Kong Express, and Hong Kong Airlines were unavailable. HKExpress cancelled some flights on 20 July. Jeju Air and Spring Japan experienced problems. Jetstar Japan cancelled many (mostly domestic) flights. Some of the self-check-in kiosks in Singapore Changi Airport were affected, delaying and forcing airlines to switch to manual check-in, and Singapore Airlines and Scoot reported various levels of service difficulties throughout 19 July. Cebu Pacific and Philippines AirAsia flights were delayed. Long queues formed at Ninoy Aquino International Airport. In Taiwan, airline system disruptions were reported at Taoyuan International Airport. In Indonesia, disruptions were reported for the check-in systems of AirAsia and Citilink. In Thailand, Thai AirAsia's reservation and check-in systems were affected.

In India, the outage affected Indigo Airlines, Akasa Air, SpiceJet, and Vistara. Handwritten boarding passes were being issued during the outage. The Ministry of Civil Aviation asked and ordered the airlines as well as the airports to be compassionate and provide food as well as seats to waiting customers as needed. As of 18:14 IST (12:44 UTC), over 200 Indian flights had been cancelled; IndiGo alone cancelled 192. Airlines that relied on Microsoft Azure for their services were impacted. Air India said that none of its flights were cancelled or delayed due to the outage, attributing it to their robust cyber system.

Europe

Prague Airport in Czechia, Budapest Airport in Hungary, Bratislava Airport in Slovakia, and Schiphol Airport in the Netherlands experienced problems. Planes were not allowed to land at Zurich Airport. Near Brussels, Charleroi Airport employees manually checked passengers in, but other software alleviated problems by 10:00 (UTC+2) and there were minimal delays. ENAIRE's Aena, the Spanish national airport traffic control manager, mentioned an IT outage in their website and social media. All Spanish airports reported disruptions. Charles de Gaulle Airport and Orly Airport experienced check-in problems and suspended flights. Poznań–Ławica Airport and Warsaw Chopin Airport experienced check-in disruptions. An emergency system was activated, and check-in processes were slower. Berlin Brandenburg Airport announced that since around 07:00 (UTC+2), operational processes were affected by "IT problems at an external provider", and that they planned to stop flights until 8 UTC. While passenger handling continued with some restrictions, there were delays and airlines cancelled some flights. Several airlines (Eurowings, Ryanair, Vueling, and Turkish Airlines) in Hamburg Airport had to issue tickets by hand. Croatian and Swedish air traffic control were also disrupted.

Swiss International Air Lines had 30% of flights grounded. Lufthansa in Germany experienced problems with the "profile and booking retrieval" features of their website. Ryanair's booking and check-in services were unavailable and the airline was "forced to cancel a small number of flights", advising passengers to arrive at airports at least three hours before departure. Wizz Air said the outage put its online services offline. Dutch airline KLM suspended most operations, announcing that flight handling was impossible with the issue, and Transavia Airlines experienced problems. Finnair reported that they were having trouble sending emails and SMS messages to customers. In Greece, citizens and tourists saw delays at major airports, notably at Athens International Airport and at Heraklion International Airport. This disruption, occurring at the peak of the tourist season, resulted in chaotic scenes as passengers were forced to wδρόμιο «Ελ. Βενιζέλος» – «Δώστε μας ένα δωμάτιο να διανυκτερεύσουμε, έχουμε μωρό» |url=https://www.newsbomb.gr/ellada/story/1568041/panikos-sto-aerodromio-el-venizeloait for hours for their flights. Contributing factors included severe staff shortages and new schedules. In Heraklion, eight flights were problematic. The airport's chief, George Pliakas, indicated that flights were being manually arranged to manage the disruption, but the influx of arriving flights strained the system.

Several UK airports had problems, including Edinburgh Airport, whose departure boards froze, and Gatwick Airport, where automatic barcode scanning stopped working and had to be checked manually. Amadeus, which manages UK baggage at Heathrow, said they were affected by the IT outage. Disruption to flights was anticipated in the Isle of Man, particularly to and from the UK, but ultimately minimal.

Middle East and North Africa

Tunisia experienced temporary airport disruptions. Turkish Airlines cancelled some of its flights due to the outage.

North America

A blue screen at the Boise Airport in Idaho during the incident.

A ground stop was issued by United, Delta, and American Airlines. Flights in the air continued flying, but no new flights were to take off. Allegiant Air were also grounded by the outage according to the Federal Aviation Administration (FAA). About 1,500 flights were cancelled in the United States on 19 July due to the outage. While American Airlines, United and Allegiant recovered relatively quickly after Friday, Delta Air Lines experienced an operational meltdown which continued through the weekend.

Delta Air Lines was by far the hardest hit of the US major airlines, with over 1,200 flights cancelled on Friday, 19 July, and thousands more cancelled on Saturday and Sunday. Thousands of stranded travelers were forced to spend the night sleeping on the floor in the terminals at Delta's largest hub, Hartsfield–Jackson Atlanta International Airport (which is also the busiest airport in the world by passenger traffic). Metro Atlanta hotels and rental car companies were overwhelmed by the scale of the crisis, leaving travelers no other option but to stay in the airport. One traveler attempting to return home to Tampa (after giving up on reaching California) reported that Amtrak was charging $1,000 for a one-way train ticket from Atlanta to Tampa. Visibly distraught passengers with nowhere to go were seen trying to sleep in the airport on hard linoleum floors without blankets or food. Without warning, Delta immediately banned unaccompanied minors on its flights through the end of 23 July. This imposed considerable hardship on parents who had been counting on that service to avoid having to fly with their children to escort them to summer camp or a relative's house, along with minors who had expected to use that service to fly home.

On 20 July, Delta cancelled over 1,400 flights.

On 21 July, Delta cancelled over 1,300 flights. With so many passengers still stuck in Hartsfield–Jackson after two consecutive nights, the airport implemented traveler assistance measures such as a "concessions crisis plan" and a plan to reunite passengers with their checked baggage. However, passengers in Atlanta on that date continued to report "jam-packed" conditions and witnessing "heartbreaking" scenes in the terminals.

On 21 July, Delta CEO Ed Bastian apologised to customers in a statement and revealed that the outage had affected one of Delta's crew-tracking software programs. According to Bastian, the software was "unable to effectively process the unprecedented number of changes triggered by the system shutdown". According to Delta CIO Rahul Samant, the program was restored and brought back online at around 11 a.m. on 19 July, but was overwhelmed by the backlog of updates awaiting processing and had been trying to catch up ever since. After the ground stop left too many crew members in the wrong places, Delta was struggling to assemble enough pilots and flight attendants at airport gates to operate scheduled flights. Many flights were repeatedly delayed and finally cancelled because the one or two crew members who made it to the gate for a particular flight kept hitting their legal flight time limit before the airline could finish fully staffing the flight, and this caused the crisis to snowball as those crew and their aircraft were now in the wrong place for the following day's flights. A similar issue had been blamed for the 2022 Southwest Airlines scheduling crisis. That same day, US Secretary of Transportation Pete Buttigieg said on social media that the US Department of Transportation had received hundreds of complaints about Delta, and reminded the airline of its legal obligations to affected passengers.

On 22 July, Delta cancelled over 1,200 flights. On 23 July, the Department of Transportation announced the launch of a formal investigation into Delta's treatment of passengers. Delta promised to cooperate but stated that it was focused on its recovery. Senator Maria Cantwell in her capacity as chair of the Senate Committee on Commerce, Science, and Transportation wrote to Bastian to express her concern for Delta passengers.

United Airlines' smaller number of cancellations had a significant impact on its hubs. For example, San Mateo County hotels around San Francisco International Airport rapidly filled up with travelers on 19 July. Guests reported difficulty with checking into the local Marriott hotel because Marriott International was also recovering from the outage.

Southwest Airlines (the third largest US major airline by domestic passengers) was entirely unaffected. A Southwest spokesperson confirmed that the airline had seen no impact from the CrowdStrike outage but refused to confirm speculation among aviation industry analysts that it had been shielded by its notoriously outdated software.

The flight delays meant that many people who had traveled to the 2024 Republican National Convention—which concluded the day the outages started—were stuck in the convention's host city of Milwaukee, Wisconsin.

Montréal–Trudeau International Airport and Toronto Pearson International Airport were affected in Canada, and Porter Airlines cancelled all flights. Vancouver International Airport was also reportedly affected in Canada, although it was unclear whether this was directly related to the global outages.

Finance

A payment terminal affected by service disruption from the incident in India

Microsoft and CrowdStrike stocks fell as a result of the outage. CrowdStrike's stock fell over 11% on 19 July, although Microsoft was down less than 1%.

Banks that were affected included Chase, Bank of America, Wells Fargo, U.S. Bank, Capital One and Charles Schwab in the US; RBC and TD Bank in Canada; Capitec Bank and other South African banks; several Israeli banks; and several banks in the Philippines, including RCBC, Metrobank, LandBank, BDO, UnionBank, BPI, and PNB. E-wallets such as Maya and GCash also experienced problems in the Philippines. The website and mobile banking application of DenizBank in Turkey could not be accessed. Visa was affected. Numerous Singaporean companies, including Singapore Exchange (SGX) and DBS Bank, reported various levels of service difficulties throughout 19 July.

In India, the Reserve Bank of India said that only 10 banks and NBFCs were affected by the outage; few banks use CrowdStrike tools and many banks' critical systems do not run on the cloud. NSE, BSE, and India's largest bank, State Bank of India, said they were unaffected.

In Brazil, Bradesco Bank confirmed it was affected. During the morning customers were able to login, but at 12:00 UTC the bank disabled the login button.

New Zealand banks ASB and Kiwibank were affected, while Australian banks Westpac and ANZ also had problems. Apps of Australian banks NAB, Westpac, ANZ, Commonwealth Bank, Bendigo Bank, and Suncorp were affected.

The London Stock Exchange, while operating normally, was unable to push news updates to its website. English gambling company Ladbrokes Coral and English supermarket chain Morrisons also reported problems. Polish banks, including Santander Bank Polska, ING Bank Śląski and mBank, encountered issues related to the outage. Santander BP's helpline, video, and chat services were affected. PKO Bank Polski clarified that its iPKO and IKO services were stable, but other banks faced difficulties. In Finland, OP Financial Group reported minor disruptions on investment partner and stock savings accounts. Sense Bank in Ukraine experienced outages due to the update.

Paraguayan banks Ueno and Banco Continental were affected; their customers were unable to log in.

Government

The United States Department of Homeland Security, NASA, Federal Trade Commission, National Nuclear Security Administration, Department of Justice, and Department of Education were affected, and the Department of the Treasury and Department of State reported minor disruptions. The Department of Veterans Affairs and Department of Energy experienced disruptions, but it is not currently known if they are related to the incident. DMV agencies for the states of Georgia, Kansas, Missouri, North Carolina, Tennessee, and the District of Columbia were affected. Ted Wheeler, the mayor of Portland, Oregon, declared the outages to be a city emergency. Election and voting registration databases in Arizona, South Dakota, Texas and the state of Washington were affected. The website for the city of Sioux Falls, South Dakota, went down.

In the United States, there were outages in 911 service or disruptions in 911 call centres' operation in some parts of Alaska, Arizona, Florida, Iowa, Indiana, Kansas, Michigan, Minnesota, New York, Ohio, Oregon, Pennsylvania, and Virginia. 911 was down for all of New Hampshire. In addition, Alaska was experiencing issues with non-emergency call centres. Many call centres switched to working backup systems.

The CM/ECF and PACER computer systems used by the US federal courts were unaffected. However, several state courts reported problems with their computer systems, including courts in Alaska, California, Delaware, Idaho, Kansas, Maryland, Massachusetts, Michigan, Nevada, New York, and Pennsylvania. In New York City, courts and correctional facilities were disrupted, delaying a hearing in the trial of Harvey Weinstein for sex offenses.

Government websites in the Philippines, such as the website of the House of Representatives of the Philippines, were down due to the outage.

In Canada, services in Toronto were affected, and Canada Child Benefit payments were delayed. New Zealand Parliament had problems. Sunshine Coast Council was one of several councils affected in Australia. The National Security Authority spokesman confirmed several institutions in Slovakia were affected.

The fire department in Copenhagen, Denmark, was unable to receive automatic fire alerts from buildings.

Ground transport

Traffic disruptions were reported at the US–Canada border, including long delays at the Ambassador Bridge and Detroit–Windsor tunnel between Ontario and Michigan. The Canada Border Services Agency blamed a partial outage of its telephone reporting system which was later resolved. There were long delays and police advised motorists to avoid the area. The Washington Metro Area Transit Authority suffered minor service delays in the early morning in America; their website/live tracking was unavailable until around 9:30 am on 19 July. The Massachusetts Bay Transportation Authority in Boston, as well as the Metropolitan Transportation Authority in New York, lost vehicle tracking and arrival notices for passengers. Most of North American freight and passenger train operators went largely unaffected aside from some technical issues within Union Pacific and Canadian Pacific Kansas City, Amtrak was mostly unaffected aside from issues with credit card processing during the morning.

Malaysia's railway operator, Keretapi Tanah Melayu, confirmed that its KITS ticketing system was experiencing technical issues. Transport for Ireland said its apps were down due to the outage. Ireland's Road Safety Authority said it was experiencing "significant disruption" to its National Car Test (NCT) centres. In Singapore, the entrance and exit gantries of over 185 car parks managed by the Housing and Development Board (HDB) were affected.

Fuel stations have also been affected in Australia, with people stuck at fuel pumps unable to pay for petrol because payment systems were not working. Auckland Transport's HOP card in New Zealand had problems. Australian freight train operator Aurizon was affected. Regional trains in New South Wales, Australia on the Hunter Line and the Southern Highlands Line were cancelled or delayed with the Regional Bus and Train network in Victoria operated by V/Line having all lines suspended.

UK rail companies were also affected. Cab riders in London could not pay with credit or with debit cards and thus required cash. In Sweden and Belgium, tickets for public transport could not be sold, and Keolis Nederland experienced issues.

Healthcare

Many hospitals across North America paused non-urgent surgeries and visits. Some affected hospitals, while remaining open, had limited, if any, access to patient records. In the United States, Memorial Sloan Kettering Cancer Center postponed all procedures that required anaesthesia, the Mass General Brigham hospital system cancelled all non-emergency procedures and medical visits, and the Cincinnati Children's Hospital Medical Center was also affected. University Health Network experienced technical issues in Canada, saying hospitals' clinical activity would continue but warning that appointments may be delayed. A number of other Canadian hospitals faced difficulties, with Newfoundland and Labrador Health Services activating contingency plans as patient record systems were affected. LabCorp and Quest Diagnostics were impacted by the outage.

Britain's National Health Service (NHS) said that the issues are "causing disruption in the majority of [English] GP practices", with some of its services, such as GP surgeries, which rely on a software product called EMIS Web, unable to view and manage medical records, issue and manage prescriptions, or make appointments. Manx Radio reported that GP surgeries were affected in the Isle of Man. The London Ambulance Service experienced an unprecedented surge in 999 and 111 calls following the outage, responding to 4,500 emergency calls by 17:00 (BST).

Two-thirds of Northern Ireland's general practices (GPs) were affected. At hospitals radiation therapy, bookings for operating theatres, and staff rosters are also affected.

In Belgium, FPS Public Health said the outage disrupted new-patient admissions in two hospitals, which activated their emergency IT plans. Two hospitals in Lübeck and Kiel, Germany, cancelled non-emergency operations. The Spanish regional governments of Aragon, Basque Country, Castilla-La Mancha, Catalonia, and Galicia reported problems with their healthcare services. Hospital Fernando Fonseca in Portugal reported problems, while the Catholic Health system in New York experienced outages that caused delays in services.

In the Netherlands, the outages affected two hospitals—the Scheperziekenhuis in Emmen and the Slingeland Ziekenhuis in de Achterhoek—and numerous emergency aid stations were also affected, including those in Emmen, Hoogeveen, and Stadskanaal.

Systems in Wesley Hospital and St Andrews Hospital in Brisbane, Australia, were affected.

The Central Health information system in Croatia was affected, although it was clarified that it was due to a concurrent issue tied with moving their servers to a new location.

In Israel, Magen David Adom and its emergency service hotline was affected. Hospitals including Sheba Medical Center, Rambam Hospital and Laniado Hospital were experiencing problems that led to longer waiting times and delayed surgeries.

The pharmaceutical company Krka in Slovenia suffered a full production outage and sent its workforce home.

Media and communications

Numerous American TV stations were unable to broadcast because of the global outage. KSHB-TV, one of the affected stations, had to resort to airing national news via Scripps News. ESPN was unable to air the morning editions of SportsCenter on the morning of the outage in America, instead airing ESPN Radio's Unsportsmanlike, simulcasting with ESPN2. ESPN and ESPN2 later simulcasted Get Up! and First Take in place of SportsCenter, albeit without on-air graphics or b-roll. Various Paramount channels were also affected including Nicktoons (with its West Coast feed switching to an old emergency feed), TeenNick, BET Her, and most channels on the Pluto TV service. MeTV Toons was sent off the air for five and a half hours. Mercedes AMG PETRONAS F1 Team also suffered issues on the Friday of the Hungarian Grand Prix, with a Mercedes spokesperson confirming that the team had to manually address the problem on every computer it used. The issue also affected their engine customers, McLaren, Aston Martin and Williams. Many video screens in New York City's Times Square turned off.

When some companies let their employees go home early as a result of the incident, the topic "Thank you Microsoft for an early vacation" momentarily became Weibo's most popular term. Universal Studios Japan announced that they would not be selling tickets via ticket booths over the weekend due to the outage; however, tickets would still be sold online or via designated ticket sales sites.

Vodafone experienced outages. The issue affected the office laptops of DPG Media Belgium – which impacts JOE and QMusic Radio, banks, post services, and government agencies. Telephone communication with the urban services in Antwerp were also affected. The Centre for Cybersecurity Belgium stated that the impact in Belgium was limited. Sky News was unable to broadcast live in the UK, as was the BBC's CBBC, a free-to-air children's television channel. Irish national broadcaster RTÉ said its newsroom was hit by "intermittent internet outages" with minimal impact to output. The Canadian Broadcasting Corporation was also impacted.

Several French TV channels affected by the issues include TF1, TFX, LCI and Canal+ Group networks. Phone and internet service provider Bouygues Telecom has also announced the unavailability of its customer service as a result of the outage. The operations of the 2024 Summer Olympics, scheduled to start the following week in Paris, France, were also affected. The outage occurred a day after the Olympic Village opened and organisers were processing the arrivals of athletes and delegates. The organising committee said that a contingency plan was activated and that only the delivery of uniforms and accreditations were affected. The incident slowed down the operations, with the accreditation desk at the press centre closed and security checks done manually using a list of names.

IT workers and the BPO industry were affected in the Philippines. Numerous Singaporean companies, including SPH Media, Singtel, and M1, reported various levels of service difficulties throughout the day on 19 July.

Australian media firms affected by the issues include the ABC, SBS, Seven Network and Nine Network. Ticketing at Docklands Stadium for Friday night's Australian Football League match between the Essendon Bombers and the Adelaide Crows was affected.

Israel Post was affected and Ukrainian Nova Poshta experienced outages. In the US, UPS and FedEx were affected.

Sim racing service iRacing was also affected by the outage in America. Various Korean online games, like Black Desert Online, Ragnarok Online, and Ragnarok Origin shut down.

Amazon Web Services, eBay, Google Cloud, Instagram, and Plenty of Fish were also affected.

Retail

The Vanden Borre retail website in Belgium experienced downtime due to the outage.

Notice in a Woolworths supermarket in New Zealand

Supermarkets in the Philippines were affected due to crashed POS systems. German supermarket chain Tegut closed some of its stores. Customers have also experienced payment issues at Foodstuffs and Woolworths supermarkets in New Zealand. British grocery chain Waitrose could only accept cash from customers. Australian retailers and fast food chains are also hit by the outage, causing self-checkout and online order systems to be out of service. Supermarkets affected included Woolworths and Coles.

The international chain Starbucks's mobile application was limited to basic viewing of accounts made before the update, alongside ordering being limited to in-store purchases only, and cash register software crashes.

In the United States, system issues caused by the incident forced sporting goods retailer Dick's Sporting Goods to close some of its stores and caused temporary outages to its website.

Convenience store chain 7-Eleven experienced issues at its Speedway branded locations still running on Speedway's legacy software using BlueCube and Radiant Site Manager dating from the days Speedway was owned by Marathon Petroleum Corporation, with some stores unable to accept credit or debit transactions while others were closed outright. 7-Eleven, which was in the process of converting Speedway locations to its proprietary RIS software, did not experience issues at Speedway locations that already switched to RIS, nor did it experience issues at standard 7-Eleven or Stripes Convenience Stores locations.

In Norway, the pharmacy chain Apotek1 and the insurance company Tryg were compelled to suspend services; Vitusapotek and Boots were also affected. Beyond these disruptions, Norway experienced minimal impact owing to CrowdStrike's limited market share in the country.

Amazon suffered disruption to its warehouse operations and internal software. An app used in Amazon warehouses to manage schedules and time-off requests called 'A to Z' was taken down by the outage and an internal service called 'Anytime Pay' became unavailable to employees. Operations were briefly halted at some sites, while Amazon's trucking operations were disrupted, with drivers saying a platform they use called 'Relay' suffered issues meaning they were briefly unable to pick up loads at warehouses.

Other sectors

The outage affected terminal operations at DCT Gdańsk, a major container hub in the Baltic port of Gdańsk in Poland. Shipping ports in the US were unaffected for the most part, although the Port of Houston (which handles the most foreign tonnage) closed briefly.

In Sweden, the Malmberget mine was evacuated as a precaution. Tickets for soccer games could not be sold.

In the United States, security provider ADT was affected.

In Germany, Tesla halted production at its Gigafactory Berlin-Brandenburg for about four hours.

Response

In a live interview on NBC's Today, CrowdStrike CEO George Kurtz apologised to the public. He said company leaders were "deeply sorry for the impact that we've caused to customers, to travelers, to anyone affected by this, including our companies". CrowdStrike warned that malicious actors might try to pose as its staff or independent researchers claiming to help fix the problem.

Political

The Australian government held a national emergency meeting to address the outage. The National Coordination Mechanism was activated; Prime Minister Anthony Albanese said, "I understand Australians are concerned about the outage that is unfolding globally and affecting a wide range of services. My Government is working closely with the National Cyber Security Coordinator". He later said, "There is no impact to critical infrastructure, government services, or Triple-0 services at this stage". Victorians were advised to call Triple-0 if a fire alarm sounds or smoke is detected, as some automatic alarms in buildings may not automatically call fire services due to the outage.

United States president Joe Biden's administration was in touch with CrowdStrike to offer assistance, and on 23 July, Kurz was invited to appear before Congress to explain how the outage occurred and what Crowdstrike is doing to prevent future incidents.

The UK government's COBR committee met to discuss the incident.

India's Minister of Information and Technology Ashwini Vaishnav said that the government was in touch with Microsoft. The government's CERT-IN cybersecurity agency classified the incident as "critical".

In Russia, the government noted that the sanctions and boycotts placed on Russia as a result of its invasion of Ukraine in 2022 had unintentionally shielded it from the outage. Russia's Digital Communications Ministry said, "At the moment, the ministry has not received reports of system failures at Russian airports," and "The situation with Microsoft once again shows the importance of import substitution of foreign software, primarily at critical information infrastructure facilities." The Russian Federal Air Transport Agency confirmed that no domestic airlines were affected. The Kremlin stated that its systems were working as normal.

Industry

Cybersecurity consultant Troy Hunt called the incident the "largest IT outage in history" and adding: "This is basically what we were all worried about with Y2K, except it's actually happened this time". Slate described it as "Y2K Lite".

Elon Musk—CEO of Tesla, X Corp, Neuralink, and SpaceX—posted on X that CrowdStrike has been "deleted from all our systems".

AirAsia CEO Tony Fernandes demanded answers and compensation for millions of dollars in revenue he said the company had lost in the incident.

Chinese cybersecurity companies such as 360 Security, QAX and Tencent took advantage of the CrowdStrike incident to promote their own software.

News reporters have used the term "digital pandemic" to describe the outage.

Criminal

Governments worldwide and cybersecurity agencies warned of digital phishing scams following the incident. Cyber criminals started sending phishing emails purporting to be CrowdStrike support and impersonating CrowdStrike staff in phone calls shortly afterward.

Analysis

Cause

The 19 July update was an instance of a template that was tested and released in March 2024 as part of an update to the Falcon Sensor software. This new instance, Channel File 291, passed validation due to a bug in CrowdStrike's content verification software.

Centralisation and homogeneity

The outage raised questions about oligopoly and centralisation in the information technology sector. The majority of the world's computers use Microsoft Windows, creating a monoculture that reduces resiliency. Ciaran Martin, a cybersecurity expert, said, "This is a very, very uncomfortable illustration of the fragility of the world's core internet infrastructure". Critical infrastructure expert Gregory Falco said, "Cybersecurity providers are part of this homogenous backbone of modern systems and are so core to how we operate that a glitch in their operations will have similar impacts to failures in systems that are household names". Security experts suggested more redundancy to avoid single points of failure, wider use of decentralised and heterogeneous federated systems, and public anger at the failure of political leaders to regulate for diversity and competition.

IT practices

Experts speculate that the update was not put through routine patch management procedures (testing the update in a sandbox) to verify there were no problems.

Mandating disclosure of breaches and vulnerabilities has also been suggested. In an interview with Wired, cybersecurity consultant Jake Williams said that this outage has "shown why pushing updates without IT intervention is unsustainable," and that "people may now demand changes in this operating model."

Operating system design and antitrust enforcement

Microsoft blamed a 2009 antitrust agreement with the European Union that they said forced them to sustain low-level kernel access to third-party developers. The document does not explicitly state that Microsoft has to provide kernel-level access, but says Microsoft must provide access to the same APIs used by its own security products. The EU rejected the allegations.

In Linux, it is possible to use eBPF instead of kernel modules to program this type of software.

Since macOS Catalina (2019), this type of software can use the Endpoint Security Framework instead of a kernel extension, and this approach has been gradually enforced.

See also

• IT risk management
• Patch management
• Security management
• Internet outage
• Google services outages
• 2021 Facebook outage
• 2022 Rogers Communications outage
• 2023 Optus outage
• Year 2038 problem