Preface
As soon as I opened my eyes in the morning, I saw a bunch of emails from Cloudflare:
From the emails, there were 5 attack notification emails in total. In chronological order, their contents are as follows:
1、
2、
3、
4、
5、
Being attacked for the first time, feelings of curiosity and surprise actually dominated, but having my free quota of 100,000 Workers GET requests for today completely used up left me a bit speechless. Let's not think about that for now, and instead review the process of this attack.
WAF Log Analysis
First, let's look at the time the initial warning email was received:
2:16 AM, so let's first check if there were any anomalies around that time in the WAF logs:
Click on the number 86.4k to enter:
Although 5 types of rules were triggered, I took a look and found that these packets did not differ much in content. They were just POST requests accessing my blog's homepage, except they were initiated simultaneously from all over the world within 15 minutes. Therefore, I will take the top two types of attacks by volume as examples below to look at their general content.
The top-ranked attack type came from the third email
The rule ID is: "6e3ccc23900c428e8ec0fb8a3a679c52", and the rule description is: "Requests coming from known bad sources".
Click on any one of them to view the details:
From the red box, nothing too unusual can actually be seen. The UserAgent was written very carefully, similar to a normal browser's UA, unlike some crawlers that are written very simply or are even completely empty.
The second-ranked attack type came from the fifth email
The rule ID is: "0a07c24f3cd44a57a5c19b73d2f294d7", and the rule description is: "HTTP requests trying to impersonate browsers".
Click on any one of them to view the details:
The content is similar to the first type of attack, and the content of the remaining 3 types of attacks is also the same. It is estimated that being classified into 5 types is based on Cloudflare's own signature database and its own analysis algorithms; to human eyes, they probably all look the same.
Because the free version's statistics are too coarse, I cannot see the exact start time of the attack. The WAF attack log also doesn't have an option to go directly to the last page, so I have to click through it myself. At 25 items per page, with hundreds of thousands or millions of items, it would take forever to flip through them. Therefore, I can only estimate the attack duration from the earliest email notification time of 2:16 and the time of the last request filtered by rule ID in the logs. The following 4 are the first 4 attack notification emails, all starting at 2:16:
It can be seen that these 4 types of attacks all started at 2:16 and ended in the interval between 2:28 and 2:31 respectively.
The 5th attack notification email was received at 2:29, and it also stopped at 2:30, which had the shortest duration:
From the above start and end times, it can be seen that although there are 5 categories, they all ended around 2:30, which should be under unified control (start times are hard to control uniformly, but end times are easy to control, so as the saying goes: "We do not seek to be born on the same day, but we hope to die on the same day"?).
Analysis and Logs Section
Web traffic:
Threats:
Workers:
Site-wide rate limiting
I have set up site-wide rate limiting, let's see if it took effect.
However, these only take effect for a large number of requests initiated by the same IP, and they must access the URI specified in the settings. In the attack, there were also many instances where a single IP only sent 1 request, which is ineffective against this.
Chaitin WAF data
I don't understand Chaitin's statistics; the request count statistics between the two pages differ quite a bit. But since it's the community free version, there's not much to complain about...
Overall, Chaitin did not block a single incoming request, which is reasonable because, as we saw earlier, looking at a single request alone, these requests don't show any obvious issues. Furthermore, although I also configured high-frequency access blocking on Chaitin:
I wronged Chaitin; it was my own fault for whitelisting the IP of the device where the Cloudflare tunnel is deployed.
Summary
Although I don't know who is so bored as to launch a distributed DDoS attack on my crappy little site, it doesn't stop me from using this as a case study for a brief analysis.
If I were still using Tencent Cloud CDN deployment, according to my usual configuration strategy:
This situation would have triggered one of these two burst usage limits, and according to my settings:
Once the 5-minute mark is reached, the CDN service would be shut down directly and return a 404, so the consumed bandwidth and the number of HTTPS resource pack requests would definitely not reach Cloudflare's final 35GB and 11 million SSL requests, but the trade-off is that the website would be inaccessible before the 60-minute unblocking time is up.
So the final conclusion is: if I hadn't relied on Cloudflare's globally deployed IP Anycast architecture to handle this distributed DDoS attack this time, but had still used Tencent Cloud CDN, there would have been an hour of downtime. And this is assuming the attack only lasted 15 minutes and occurred only once; if this kind of attack happened once every hour, my website would be permanently inaccessible (Cloudflare is awesome).
However, this also exposed that my current home data center still lacks a traffic traceability solution, which I need to think about carefully; otherwise, when the next problem occurs, there will still be no way to trace it locally. Should I get a Colasoft solution suite?
Note 1: Technically speaking, Cloudflare's globally deployed IP Anycast architecture is indeed too easy to use against this kind of globally initiated distributed DDoS attack, as the traffic distributed to each edge node is pitifully small.
Note 2: If I had manually enabled the "Under Attack" button during the attack:
Would it have avoided consuming my Workers' free quota? Of all times, the attack started at over 2 AM; it must have been intentionally timed for when I was asleep. So insidious...
It seems that using the free quota of 100,000 requests per day on Workers to optimize blog access is still a bit fragile, and I need to explore new ways to optimize access.
Aren't home data centers usually deployed on the intranet? For these relatively important services, I usually either don't map them to the public network or access them via VPN, as I feel very uneasy about exposing them directly to the public network.
Generally, what people on the internet call a ”home data center” is just a euphemism; for example, just a single NAS plus public port mapping or intranet penetration can also be called a ”home data center.” Strictly speaking, these can only barely be considered a ”pseudo-home data center” in a broad sense. From a professional perspective, data centers are classified by usage scenarios into those providing internal services (such as access only for internal employees of an organization) or those providing services to the public (such as the financial and tax industries facing public access). But the ”home data center” I am talking about is professional, which itself is a complete set of solutions including security and caching: from DNS resolution - DDoS mitigation - web security filtering - access speed optimization - intranet penetration - secondary filtering by web application firewall - load balancing (application hot standby), and it also includes data synchronization for multiple WordPress sites, as well as same-city disaster recovery to enable a cloud server mirror site with one click when there is a power outage at home. All of these combined are what I call a ”true home data center,” not just the concept of mapping to the public network or throwing it directly onto the public network as you mentioned. Or rather, the purpose of my home data center solution is precisely about how to safely and legitimately put home applications on the public network (as Ng Man-tat said in Shaolin Soccer: I'm not just a simple cripple~). You should know that the performance of any idle device at home is much better than the stingy 1-core/2-core, 1GB/2GB RAM of cloud servers, and the only cost is just a tiny bit more on the monthly electricity bill.
If the performance is sufficient, after all, the upper limit of human technology is right here, and power consumption cannot be low anyway. For a device with 15W power and an annual electricity bill of 60, its performance is about the same as those cloud services that cost 98 a year. It's nothing more than having larger memory and hard drive space. But considering the cost of the hardware itself and the cost of future upgrades (as for idle equipment, it could originally be sold second-hand to recover some costs). Within a certain intensity of use, self-hosting might actually not be as cost-effective as using some cloud services.
Well, technically that's true. However, buying a cloud server requires ICP filing in China, and access speeds from abroad aren't necessarily fast. Using your own equipment wins in terms of simple deployment, flexibility, and strong controllability. As long as the issue of slow access from China can be resolved, for those with some hands-on ability, there is still a strong desire to tinker with it.