Contents
Project Background
Only 2 or 3 friends were looking at my emby before, and I didn't think too much about it, so I used an unregistered domain name and directly mapped the port on iQIYI to the intranet http port of emby installed on the NAS. This was nothing, but since the telecom incident (see previous article for details:Home Data Center Series Home Broadband Site Building Considerations with Public IP), and then I thought that unregistered domain names plus http plain text were too dangerous, so I came up with the idea of optimizing the emby publishing method; then I thought that IPv4 public network addresses are so precious now, and I have to be on guard against the day when the telecommunications company forcibly reclaims IPv4 public network addresses under the banner of complying with the country's "network power" policy, so I simply optimized the overall architecture of my home data center and prepared in advance for the construction of websites with only IPv6 public network addresses in the future; finally, I thought, since the project scale is so large, I might as well take the opportunity to sort out the traffic access logic of the home data center, and as a result, I came up with this project: Home Data Center v4v6 Dual Stack Network Architecture and Application Access Process Optimization.
I just wanted to write a casual article, but I accidentally made the project bigger? ? I was caught off guard. Is this "a murder caused by an emby"?
Original environment analysis
The original environment flow logic is quite confusing, as shown below:
Tencent Cloud CDN's back-to-source request reaches iQIYI via Telecom Line 1, Cloudflare's tunnel access request reaches iQIYI via Telecom Line 3, and Emby's http request reaches iQIYI via Telecom Line 2. The key is that there is no unified traffic processing logic for the access traffic entering the home data center through iQIYI soft router.
As can be seen from the above figure, there are three ways for access traffic to reach intranet applications:
1. AiKuaiShang directly maps the port to the application (representing the application emby), and it still runs in plain text http (Telecom saw it but didn't say anything).
2. The AiKuai port is mapped to the specified port (https) of the v4 gateway (that is, the Baota Linux panel), and then the v4 gateway unloads SSL and directly points to the application through the reverse proxy.
3. AiKuai port is mapped to WAF, and then WAF performs security check on the access request and points to the application through reverse proxy. The application here also includes v4 gateway. Why? Because I used to deploy some applications directly on the Baota panel for convenience.
To sum up, the traffic logic of the entire network is actually a mess. Of course, this is mainly a historical problem, and it can't just be said that I'm lazy.
In addition to the messy traffic logic, I didn't use IPv6 at all, which wasted the v4/v6 dual stack environment. At the same time, there were no adequate security plans. For example, what should I do if I get attacked by DDoS? How to divert the attack traffic to traffic cleaning (Cloudflare)? In short, there are many problems.
Optimize the logical topology after transformation
After some work, the optimized and transformed logical network architecture is shown in the following figure:
Inbound access traffic line allocation
1. Tencent Cloud CDN back-to-source traffic enters from Telecom Line 1
To achieve this goal, use the "Advanced Application" - "Dynamic Domain Name" function of iQiyi to update the public IPv4 address of Telecom Port 1 to Domain Name B in real time:
Point the back-to-source address of the corresponding accelerated domain name on Tencent Cloud CDN to domain name B:
2. Cloudflare's tunnel access traffic enters from Telecom Line 2
To achieve this goal, we use the "Flow Control Diversion" - "Port Diversion" function of iQIYI to specify the device where the cloudflare tunnel is deployed to go out of the Telecom 2 line, so that cloudflare will connect with the IPv4 public network address of Telecom 2 port:
3. Alibaba Cloud CDN's IPv6 back-to-source traffic enters from Telecom Line 3
This goal is achieved through the "Network Settings" - "IPv6" - "IPv6 Settings" page of iKuai, which supports the function of specifying the activation of IPv6 lines in a multi-line environment, thereby enabling IPv6 on Telecom Line 3, so that IPv6 traffic can only enter from Telecom Line 3:
Because IPv6 uses public network routing, the original IPv4 port mapping function has become invalid, and iKuai does not currently support the port forwarding function of IPv6->IPv4 addresses. Therefore, IPv6 back-to-source traffic needs to be implemented in conjunction with iKuai's dynamic domain name function to reach the v4/v6 gateway (the v4/v6 gateway can also run the dynamic domain name function itself, which is more accurate. iKuai obtains the IPv6 address of other devices depending on the situation. If it is a stateful IPv6 address, the accuracy is no problem, but if it is stateless, it can only be obtained through the mac address of other devices, which sometimes fails).
From a security perspective, you can enable the IPv6 firewall function on iKuai, specifying that only requests that match the v4/v6 gateway IPv6 address suffix and a specific target port (such as a reverse proxy port, tcp, udp port forwarding port) can pass through, while requests to access other internal IPv6 public network addresses are blocked, which can greatly improve the security of the internal network IPv6 address. This goal is achieved through the "Security Settings" - "ACL Rules" function of iKuai:
4. Other inbound traffic line allocation
The access entrance of emby is now assigned to Telecom Line 3, which uses the registered domain name for access, while the data entrance of transmission, which hangs PT, is assigned to Telecom Line 1. The reason for this allocation is that the domain name accessed by emby will cause the public IP of my Telecom Line 3 to be exposed. If I suffer a large traffic attack from DDOS, I can directly disconnect Telecom Line 3. Anyway, Telecom Line 3 is mainly used by emby and IPv6 backup lines; Telecom Line 1 was originally used as the back-source line of Tencent Cloud, and there is not much traffic at ordinary times, so I just use transmission to make use of it.
The inbound traffic of clouflare accessing my com domain name goes through Telecom Line 2, which is also my Internet access line. Disconnecting Telecom Lines 1 and 3 will not affect the access to my com domain name, nor will it have any impact on my normal Internet access.
v4/v6 Gateway
Optimize the v4/v6 gateway (the original Baota Linux panel):
1. Migrate applications deployed with source code
Migrate all source code deployed applications to nginx in the docker area (for docker deployment of nginx+php, see:Docker series single container nginx, single container php (one version) multi-site sharing).
2. Enable IPv6 support for all sites
The "--with-ipv6" module is already added to the default installation of nginx in Baota Linux Panel, so you only need to add the following code to the corresponding site configuration:
listen [::]:443 ssl http2; # The specific port number can be modified according to your actual situation. For example, in this example, it is 55555As shown below:
If nginx is installed separately, you can enter the nginx file directory and use./nginx -V Check the installed modules, if any--with-ipv6, it means that the module has been installed, otherwise it needs to be recompiled and installed.
3. Enable SSL encryption on all sites
In order to cope with the increasingly severe environment in the future, all sites must enable SSL encryption (including emby). The certificate can be the built-in let's encrypt certificate of Baota Panel. If the number of hosts under the primary domain exceeds 20, ohttps can be used (see article:Home Data Center Series SSL Certificate One-Stop Management Tool OHTTPS Usage Tutorial), the v4/v6 gateway provides a unique site listening port (for example, tcp 55555, and all sites must add this listening port), and is used for port mapping of the telecom IPv4 public network address on the iKuai soft router and the port corresponding to the Alibaba Cloud CDN IPv6 back-to-source address.
All SSL decryption must be completed on the v4/v6 gateway. In theory, the intranet only runs http, except for some special applications (such as DOH).
4. Alibaba Cloud CDN IPv6 back-to-source address
In fact, why should I introduce Alibaba Cloud CDN when I already have Tencent Cloud CDN? Because Tencent Cloud CDN back-to-origin currently only supports IPv4 addresses:
This is a pitfall, but Alibaba Cloud CDN supports IPv6 back to the source:
Therefore, if there are really no IPv4 public network addresses in the future and Tencent Cloud does not support IPv6 back-to-source at that time, I will be forced to switch to Alibaba Cloud CDN.
The back-to-source address of Alibaba Cloud needs to point directly to the dynamic domain name corresponding to the IPv6 public network address of the v4/v6 gateway. This also needs to be achieved through the "Advanced Application"-"Dynamic Domain Name" function of iQIYI:
iQIYI IPv6's dynamic domain name resolution can obtain the IPv6 address of an intranet host based on the mac address and DUID of a host in the intranet, and update the host IPv6 address to the domain name A used for Alibaba Cloud CDN back-to-source. Therefore, fill in domain name A in the back-to-source address in Alibaba Cloud CDN, and point the port to the listening address 55555 of the v4/v6 gateway:
Note 1: In fact, you can also add Alibaba Cloud's address as a hot backup source station in Tencent Cloud CDN, so that if the IPv4 public network address is suddenly withdrawn one day, it will not affect the access to my domestic domain name.
Note 2: For IPv6 and IPv6 dynamic domain name configuration of iQiyi Router, please refer to:iQiyi soft router series iQiyi IPv6 function configuration tutorial.
5. iKuaiSoft Router Port Mapping
Delete all port mappings of the iKuaiSoft router's IPv4 public network address that directly point to the home data center, and create only one port mapping that points to the site listening port of the v4/v6 gateway. In this case, it points to TCP port 55555.
Note: The 55555 port pointed to by Tencent Cloud CDN back-to-origin is actually the port mapped by the port on Telecom Line 1 configured on the iQIYI router. In fact, the port on the v4/v6 gateway does not have to be 55555, but the tcp 55555 port pointed to by Alibaba Cloud CDN back-to-origin is the port on the v4/v6 gateway. For the sake of uniformity, simply set the ipv4 and ipv6 listening ports on the v4/v6 gateway to 55555.
6. Deploy v6–>v4 port forwarding on the v4/v6 gateway
For non-http applications (such as the World of Warcraft server), they cannot be deployed through reverse proxy, but can only be forwarded directly from a specific port of the IPv6 public network address to a specific port of the intranet IPv4 private address using port forwarding. This requires the deployment of lucky (for detailed lucky deployment configuration, see:Docker series uses Docker to build a simple public network IPv6 access gateway based on lucky).
7. v4/v6 gateway next hop direction
All inbound access requests, after SSL decryption by the v4/v6 gateway, need to be directed to the next-hop WAF device for security check through reverse proxy (for the reverse proxy configuration steps of the Baota Linux panel, see:Linux panel series configure reverse proxy and use non-443 port for publishing).
Although in principle all incoming requests received on the v4/v6 gateway should be directed to the lower-level WAF for inspection through reverse proxy, some special applications need to bypass WAF (such as emby's access traffic). This is because the traffic returned from the emby server is too large and all passes through WAF, which I guess WAF can't handle. In fact, the best way should be DR mode, that is, the traffic returned from the emby server directly bypasses WAF and returns to the v4/v6 gateway, but I think it's not cost-effective to do so just for one application, so I can only endure it for the time being.
Therefore, the final compromise is that emby's access request is directly directed from the v4/v6 gateway through the reverse proxy to the emby server, bypassing the WAF.
8. Reinforcement of the v4/v6 gateway itself
Actually, nginx itself needs to be reinforced, but firstly, after passing through CDN, the IP address of the source station is not exposed, and secondly, nginx on the v4/v6 gateway only does SSL decryption and reverse proxy (the source code deployed applications have been migrated to the docker area NGINX), so there is no need to toss. The only thing you can do is to configure a fake default site to deal with IP scanning and avoid leaking the site domain name. There are many tutorials on the Internet, so I won’t go into details here. Remember to use a fake SSL certificate.
WAF
Since cloudflare has built-in traffic cleaning and WAF functions, I am relatively assured of access requests entering the intranet through the tunnel, which can be directly directed to the applications in the docker area. However, for requests coming from domestic CDNs, we must be very careful, so WAF is very important: all access to internal applications on the v4/v6 gateway, except emby, must first be directed to WAF and checked by WAF.
The specific configuration varies depending on the WAF. I use the community free version of Changting Leichi, which mainly performs semantic analysis and high-frequency access/attack restrictions:
For visits from com domains, especially important applications (such as blogs), I also pass WAF in order to compare with the number of visits from domestic domains and provide the final protection:
(Optional) NGINX Load Balancing
Because I have docker1 zone (macmini) and docker2 zone (inter mini host), both of which are deployed with NGINX, so there are requirements for load balancing and application high availability. As for whether to use NGINX, HAproxy or zevenet-ce for load balancing, it depends on everyone's habits.
The interface of zevenet-ce is quite good, and the community version is free. It has all the basic server load balancing functions (it's not clear if you take a screenshot, so just bear with it and take a general look):
But in the end I chose nginx's stream module to implement the 4-layer load (the 7-layer load is not available for the time being), because it feels more lightweight, and the nginx Web UI is used to solve the problem of no graphical interface:
Because the current traffic is not large, the master-slave mode is adopted to achieve high availability of the blog.
Docker series uses Docker to build a graphical nginx based on nginx Web UI
Safety Plan
This security plan is mainly aimed at the response to large-volume DDOS attacks (non-large-volume attacks need to be handled by devices such as WAF). In normal projects, if the user purchases cloud cleaning services from a supplier (operator or cloud service provider), once abnormal traffic is detected, the supplier will pull the abnormal traffic to the traffic cleaning device for cleaning, and then send the cleaned access traffic back to the user.
However, as a blogger, I don’t have the money to do these things. So what can I do? I can only try to make the best use of the existing resources at hand. The com domain name is placed on cloudflare, so it is not afraid of ddos attacks. The main thing that needs a security plan is the traffic from China that accesses the cc domain name through Tencent Cloud or Alibaba Cloud CDN. So how to operate it?
1. Use some features of CDN to resist attacks
I have to configure IP access frequency limit, 100QPS is enough for normal application:
The usage is capped. If the value exceeds the cap, CDN will be shut down directly. Here you need to set it according to the actual traffic of your website:
Disable TLS 1.0. You can also consider disabling 1.1:
2. WAF’s high-frequency access restrictions and high-frequency attack restrictions
After a preliminary screening by CDN’s access frequency limit, WAF can screen again and limit high-frequency attack behaviors.
3. Cloudflare traffic cleaning
The cloudflare free plan can perform traffic cleaning after a simple configuration, but the default CDN IP is not the preferred IP, so user access is much slower than domestic CDN. However, it is better to be slow than not open at the critical moment (and you can manually test and fill in the preferred IP)? So you can use cloudflare as a last resort (existing security policies can no longer block attacks) for protection.
How to do it? Just connect your domestic domain name to cloudflare as a custom host name:
However, it is usually divided into two resolution results: domestic and overseas: domestic access directly uses cname to point to the address of the domestic CDN manufacturer, while overseas access directly uses A record to point to 1.0.0.5:
In this way, once a DDOS attack is discovered, you can directly suspend the CNAME record, and then change the "overseas" range of the A record to "default". Subsequent new attacks will resolve your domain name directly to 1.0.0.5, and then hit cloudflare's anycast IP, thus entering the traffic cleaning range of cloudflare.
However, this method is ineffective against existing attacks, because others have already launched the attack on your current IP through domain name resolution. So for me, there is one more step, which is to disconnect Telecom Line 1 and redial. Due to the dynamic IP mechanism, Telecom Line 1 obtains a new IPv4 address, and subsequent attacks cannot hit me.
Compared to purchasing expensive cloud cleaning services or buying DDOS equipment yourself, this method is slow to respond and requires manual intervention, but it is free. Of course, this set is only suitable for personal sites. If it is a commercial site, don't save these small amounts of money.
Afterword
Through this article, I found that the old saying of the ancients is true: a good memory is not as good as a bad pen. The above knowledge is commonly used in daily life, and I don’t think it’s a big deal. But when I wrote it down and needed to write some summary conclusions, I still found some details that I didn’t notice or think deeply about. Sure enough, sticking to blogging is really beneficial.
In addition: On April 17, I woke up and found that my multi-dial number was gone. Although I knew this day would come sooner or later, I didn’t expect it to be so soon. If this continues, the IPv4 public network address may soon be gone. Although it doesn’t affect me much, I still feel a little sad after so many years.
