How did Internet’s watchdog Cloudflare fail: X-Whatsapp-Canva did not work for 4 hours, years old bug found responsible

It happened yesterday evening. Rahul’s phone started behaving strangely. Scrolling to X just blank screen. Had to ask Chat GPT for the recipe to make for dinner, that too started showing “Something went wrong”. Rahul thought that maybe there was a problem with his phone itself.

On the other hand, college student Priya in Mumbai was sitting to complete her project on Canva. Suddenly it stopped moving. Turned Wi-Fi off and on, restarted the phone, nothing happened. Priya called her friend who was also facing the same problem.

Gradually news started coming in that this kind of problem was occurring all over the world. Turns out there was an issue with Cloudflare’s servers. You can call Cloudflare the watchman of the internet. Websites data passes through Cloudflare.

Latent bug was activated by small permission change

But how did all this happen? Cloudflare has released a detailed report of the entire incident in its official blog post. The company admitted that this was not a cyber attack, but there was a years-old latent bug in their system which got activated by a routine change.

• Cloudflare CTO Dan Nicht said that the company changed routine permissions in the database and wrong entries started appearing in the configuration file of the bot protection system.
• The file size doubled. When this file was distributed on the global network, thousands of servers crashed because the software could not handle such a large file.
• The file was being regenerated every few minutes, so the problem kept coming and going. Users were seeing 500 error messages. This problem continued from 5-9 pm.

Which services were most affected

Cloudflare provides content delivery networks, security, and routing services to more than 20% of the world’s websites. More than 14 million websites were affected by the outage. These included X, ChatGPT, WhatsApp, Instagram, Spotify, Canva, Cloud AI, Uber, Zoom. According to an estimate, Cloudflare provides service to every fifth website in the world.

Cloudflare will make 3 major changes after the outage…

• Will impose auto limit on the size of the config file.
• A new testing system will be created to find such latent bugs.
• The new version of the proxy service will be tested and rolled out in advance.

Cloudflare is a cloud services and cyber security company.

Cloudflare is a global cloud services and cybersecurity company. It provides datacenter, website and email security, data loss prevention, and cyber threat protection.

The company describes itself as “the immune system of the Internet.” Meaning its technology sits between its clients and the rest of the world and blocks billions of cyber attacks every day. Besides, with the help of its global infrastructure, it also makes internet traffic faster.

The company earns more than 500 million dollars (about Rs 4.42 thousand crore) every quarter. It has around 3 lakh customers. The company operates in 125 countries including China.

There is more news…