Contents
Preface
In the previous article (see:Cloudflare tutorial series for home data centers (Part 2) Introduction to the functions of each technical node in the CF overall solution traffic sequence), I introduced what CF does from the time when a user's access request "enters CF's edge network" to the time when the request is "prepared to be sent to the source server". However, before "entering CF's edge network", how does CF allow the user's access request to enter the edge network? And after "preparing to be sent to the source server", how does CF send the user's access request to the source server?
This article is to explore the above two issues.
How to access CF's edge network?
What is edge networking?
CF's edge network is a distributed network architecture designed to optimize the transmission and processing of Internet traffic by deploying data centers and servers around the world.
A brief description of the CF edge network is as follows:
- Global Location
• describe：Cloudflare has deployed data centers in more than 200 cities around the world, covering more than 100 countries and regions (except a few special countries).
• use: Ensure that users can enjoy fast and reliable network services no matter where they are. - Low latency
• describe: Reduce the distance and time of data transmission by placing content and computing power at the edge closer to users.
• use: Speed up web page loading and application response time, and improve user experience. - High Availability
• describe: Ensure continuous availability of services through multi-point redundancy and automatic failover mechanisms.
• use: Even if some nodes fail, traffic can be automatically switched to other nodes to ensure uninterrupted service. - safety
• describe: Integrate multi-layer security protection measures on the edge network, including DDoS protection, Web Application Firewall (WAF) and malware detection.
• use：Intercept and filter malicious traffic close to the source of the attack to protect the security of websites and applications. - Edge computing
• describe: With Cloudflare Workers, developers can run serverless code on edge nodes to process requests and perform computing tasks.
• use：Preprocess user requests before they reach the source server, reducing server burden and improving application performance.
In layman's terms: the edge network can be understood as the edge part of the entire CF network, which is very close to the actual access users and the source server. By deploying various CF services to the edge network, users' access requests can quickly obtain the required content (for example, caching the response content of the source server in the cache of the edge network closest to the access user through CDN), and malicious access requests can be intercepted (for example, the attack can be intercepted in the WAF of the edge network closest to the malicious visitor, making it impossible for the malicious visitor to enter the CF network, let alone reach the source server), and it can also quickly return to the source (initiate a request to the source server from the edge network closest to the source server and cache its response content).
In short, CF's edge network and the various services deployed in it can greatly improve users' access experience to applications.
Note: The above description is theoretical. In fact, users of the domestic Free plan call CF "negative optimization". The reason is that CF has not really extended its backbone network into the country (the reason is obvious to everyone), but only provides services indirectly through domestic partners (previously Baidu Cloud, and now it seems to be JD Cloud), and the main service targets are paying users.
For domestic users of the Free plan (webmasters who get it for free), CF obviously doesn't take care of it: for example, as mentioned above, when returning to the source, CF should initiate a request from the edge server closest to the source server, but in reality, most of the return addresses are directly assigned to the San Jose data center in the western United States. Now the return to the source has to cross the Pacific Ocean, and the response speed can be imagined. In addition, domestic users will encounter similar situations when accessing normally (some areas are even inaccessible), which ultimately leads to the access experience being faster without CF.
So is there any way for domestic Free plan webmasters to improve the speed of their sites when accessed from China? Of course there is, and let’s listen to the next section for details.
From the webmaster's perspective: How to allow user access requests to enter CF's edge network?
Conventional method: Xiaocheng Cloud
After the domain name is hosted on CF (the configuration steps for hosting the domain name on CF can be found at:Home data center series uses domestic cloud hosting to get free cloudflare to achieve fast access to domestic data from abroad), create a new host name, the default proxy status is "proxied", which is usually called the "Little Orange Cloud" status:
In this state, when the user uses DNS to resolve the corresponding host name (such as abc.tangwudi.com in the figure above), although the resolution address I set is 110.110.110.110, the actual resolution result of the user is as follows:
In the above figure, 104.21.76.18 and 172.67.185.18 are the entry IP addresses of the CF edge network. The negative optimization I mentioned earlier also refers to the effect of directly using this place for access.
So, is there any way for webmasters using the Free plan to increase the access speed of their sites? Of course there is.
Acceleration method 1: Optimize IP
The preferred IP function of CF is mainly used to optimize network performance.
Preferred IP refers to an IP address selected and assigned through a specific mechanism (such as geographic location, latency, etc.). These addresses are usually optimized and screened to provide better connection quality and lower latency. As I mentioned earlier, domestic visitors are assigned to the data center in the western United States, but if the Preferred IP function is used, they may be assigned to the address of a domestic cloud partner, which will greatly improve the access experience. Unfortunately, Preferred IP is currently a special service for paid users, and users of the Free plan cannot enjoy it (there was a time when you could get it for free through CF's domestic partners, but it is no longer possible now. I didn't enjoy it, which is a pity).
However, there is also an indirect way to enjoy the preferred IP function (folk method), which is CF's "custom host name" function + the resolution function of domestic domain name providers + the "self-selected IP" detected by specific software:
Then use the software to select the node with the highest success rate and the lowest latency:
Configure the appropriate node IP (the first IP in the figure above) to the record value corresponding to the host name that needs to have the preferred IP in the console of the domestic domain name provider:
Note 1: Strictly speaking, the "self-selected IP" selected in this way can only be effective for a short period of time. If this method is used, frequent testing is required to maintain the availability of the "self-selected" IP, which is inefficient.
Note 2: There are many tutorials on the Internet on how to use custom host names to achieve IP optimization. If you are interested, you can search for it yourself. If necessary, I will consider using a separate article to introduce the detailed configuration steps. The main reason is that I no longer use this solution (I used it last year when I was still using the registered domain name), so I don’t have much motivation to write this tutorial for the time being (the key is not something that can be explained in a few words~).
Acceleration method 2: Workers
By using CF's Workers, you can flexibly control traffic routing and request processing through programming, optimize network performance, and indirectly achieve the effect of IP optimization. This is mainly based on the following features of Workers:
- Edge computing:
Workers run in CF's globally distributed edge data centers, processing requests close to the user. By processing requests near the user, the transmission distance and time are reduced, thereby improving response speed and reducing latency.
- Intelligent traffic routing:
Use Workers code to dynamically select the best backend server or API endpoint based on real-time network conditions and performance indicators, implement intelligent traffic routing, select the best path and server, and achieve the effect of IP optimization.
- Real-time monitoring and adjustment:
Through Workers code, network performance and server health status are monitored in real time, traffic routing is adjusted dynamically, and routing strategies are automatically adjusted based on real-time data to avoid congestion and failures, thereby improving network stability and performance.
- Cache Management:
Use Workers to control caching strategies, cache static content or pre-process dynamic content at edge nodes, reduce the frequency of requests to the original server, speed up content transmission, and reduce server load.
For detailed configuration steps of configuring workers to speed up website access, see the article:Home Data Center Series Cloudflare Tutorial (VII) Introduction to CF Worker Functions and Practical Operation, Verification and Research on Related Technical Principles of Implementing "Beggar Version APO for WordPress" Function to Accelerate Website Access Based on Worker.
Note: This part is very important. It is a very practical feature for webmasters using the Free plan. If it can be configured properly, it is equivalent to having a preferred IP acceleration function with a free quota of 100,000 requests per day. Under normal circumstances, an average personal blog cannot use up 100,000 requests (except for special circumstances, such as being attacked by DDOS, which has depressed me~). Later, I will use a special article to describe the configuration process in detail.
Visitor Perspective: WARP
CF WARP is a service that provides a more secure, private, and faster Internet connection. It started as an extension to the CF 1.1.1.1 DNS resolver and is now a standalone application and service.
A brief introduction to WARP features is as follows:
- Enhanced privacy and security
• describe: WARP encrypts traffic between user devices and the Internet, preventing data from being stolen or tampered with.
• use: Protect users' online privacy and prevent hacker, man-in-the-middle attacks and surveillance. - Improve your connection speed
• describe: WARP uses Cloudflare's global edge network to optimize data transmission paths and reduce latency.
• use: Speed up web page loading and application response time, providing a smoother Internet experience. - DNS Encryption
• describe: WARP integrates the Cloudflare 1.1.1.1 DNS resolution service to encrypt DNS queries via DoH (DNS over HTTPS) or DoT (DNS over TLS).
• use: Prevent DNS queries from being eavesdropped and tampered with, and improve Internet security and privacy.
In simple terms, WARP can be understood as a "dial-up program". As long as you can "dial through" (connect successfully), you can directly access CF's edge network in an encrypted manner. If other websites that need to be accessed are also hosted on CF, the access speed can be greatly improved. Of course, I guess the main purpose of friends who used WARP in the past was for some other additional functions, but in fact, the original function of WARP is to access CF's edge network.
In short, WARP is a means for visitors to actively access the CF edge network (for detailed configuration steps, please refer to my other two articles:Home Data Center Series Reasonable use of cloudflare WARP to improve the speed of accessing websites (desktop version)as well asDeploy cloudflare warp on the home data center series cloud host to improve network access speed (Linux cli version)).
Note 1: WARP+ (premium version) uses Cloudflare's Argo intelligent routing technology (the official version of the preferred IP) to further improve performance. Normally this is a paid feature, but it can be used for free through the Zero Trust team.
Note 2: After mid-June, the stability of WARP+ seems to be questionable (my WARP could never connect before, but WARP+ can connect stably). Sometimes it works and sometimes it doesn't. It probably depends on your luck. This is because WARP(+) is based on wireguard, and wireguard is completely incompetent in terms of obfuscation (after all, it was not designed for this purpose). Its characteristics are very obvious, so it is very easy to be targeted (refer to the "regular intermittent" packet loss of IPsec VPN used in China). It is very easy to interfere with and block it.
How to choose the appropriate back-source method
What is back-to-origin?
Origin Pull is a concept in CDN: if an application is published using CDN, when the content requested by the visitor is not available on the CDN's edge cache server (cache miss or expired), the CDN server will request the content from the origin server (Origin Server), which is called Origin Pull.
Note: CDN can be considered as a reverse proxy server with caching function enabled (essentially it is), which makes it much easier to understand:
1. If the content of the access request is cached on the reverse proxy, the reverse proxy will directly return the cached content to the visitor
2. If the content of the access request is not cached on the reverse proxy, the reverse proxy will request the corresponding content from the upstream server and return the response of the upstream server to the visitor. This step is called back-to-source on the CDN.
Optional return method
Public IP address back to source
What is public IP address back-to-source?
The Origin with Public IP Addresses function means that in CF's content delivery network (CDN), the edge server requests content from the origin server through the public IP address. This is a common configuration method in the CF origin mechanism, allowing users to configure the origin server of their website or application to a public IP address (the origin server needs to have a public IP address, either v4 or v6 address), so that content distribution and request processing can be performed through CF's edge network.
For specific configuration steps, you can still refer to the article:Home data center series uses domestic cloud hosting to get free cloudflare to achieve fast access to domestic data from abroad.
Note 1: This method is actually the back-to-source method for "Xiaocheng Cloud" mentioned earlier. The configuration is also very simple. In fact, you just need to configure the correct source server public IP address in the A record in DNS, and then open "Xiaocheng Cloud". As mentioned earlier, when using this method, you don't have to worry about the source server address being exposed, because the address that is published to the outside and resolved by the user is CF's CDN address. Of course, all CDNs are in this form, so it's nothing special.
Note 2: When using the "Orange Cloud" method for public network back-to-origin in the Free plan, you can only change the target port of the back-to-origin request through the "Origin Rules" rule (suitable for cases where the source server does not use the standard 80,443), but cannot change other parameters (such as header parameters such as host). For configuration, please refer to the article:Home data center series uses cloudflare's Origin Rules to solve the problem of having a public IP but no legal ports 80 and 443 when building a website.
Note 1: This method is only recommended for those who use dynamic blogs (because static blogs can be hosted by any free hosting provider) and the deployed cloud host is not a domestic cloud host.
Why is it not recommended to use domestic cloud hosts as the source site? Technically, it doesn't matter whether it is domestic or foreign, but don't forget the particularity of domestic policies. First, ports 80 and 443 are not available and need to be registered (other non-standard ports are not safe now). Second, the domestic supervision and crackdown on unregistered domain names is increasing day by day. The formal implementation of the SNI whitelist system is only a matter of time. In this context, why bother?
Note 2: CF supports many HTTP and HTTPS ports by default, not just 80 and 443. However, except for 8080, other ports do not support caching in the Free plan, only back-to-origin.
- HTTP
8080 (cacheable),8880,2052,2082,2086,2095 - HTTPS
2053,2083,2087,2096,8443
Additional benefit: worry-free SSL certificate service
全面转移到CF之后，最让我开心的一件事莫过于：再也不用在SSL证书上花费精力了。回想我从去年9月份开始研究建站以来，最让我烦心的一件事就是SSL证书的更新问题了(为了做各种测试，我的3级域名太多了，好几十个。。。)。最开始是折腾家里和云主机上反向代理的SSL证书(开始还不懂let’s Encrypt，都是白嫖当时腾讯云1年期的免费证书，结果直接把20个额度干满了)，到后来腾讯云CDN上的SSL证书，然后最后因为想统一管理所有SSL证书的更新又去研究ohttps～～～(参见文章：Home Data Center Series SSL Certificate One-Stop Management Tool OHTTPS Usage Tutorial), in short, it's just a hassle.
After switching to CF, once the relevant domain name has enabled "Xiaocheng Cloud", according to the default configuration of SSL/TLS, all access requests to the relevant domain name will be forced to jump to https after reaching the edge network, and the subsequent SSL certificate verification and encryption and decryption operations will be completed on the edge network. These are all completed automatically and the webmasters do not need to do anything.
Of course, webmasters can also set the communication method between visitors, CF edge network, CF edge server and origin site according to their needs:
- Close (unsafe)
HTTP is used between visitors and the CF edge network and between the CF edge server and the origin server. - flexible
https is used between visitors and the CF edge network, while http is used between the CF edge server and the origin server. - completely
https is used between visitors and the CF edge network, and https is also used between the CF edge server and the origin site, but the SSL certificate of the origin site is not a valid certificate. - Complete (strict)
https is used between visitors and the CF edge network, and https is also used between the CF edge server and the origin site. The SSL certificate of the origin site is a legal certificate.
Note 1: The "complete" method is very useful. You can use ultra-long-term, non-legal certificates (such as the 15-year certificate provided by CF) directly on the source site, which saves the source site from having to worry about SSL certificates.
Note 2: In fact, domestic CDNs all adopt the "complete" method by default, that is, they do not verify the legitimacy of the source site certificate.
Using Tunnel to return to the source
CF Tunnel (also known as Argo Tunnel) is a service that enables servers, applications, and devices to securely connect to Cloudflare's edge network without publicly exposing their public IP addresses.
A brief introduction to the CF Tunnel function is as follows:
- Secure Access:
• describe: Securely connect internal networks, applications, and devices to Cloudflare’s edge network through encrypted tunnels.
• use: Avoid directly exposing the server's public IP address to prevent external attacks and unauthorized access. - Simplify network configuration:
• describe: With Cloudflare Tunnel, there is no need to configure complex firewall rules or VPN.
• use: Simplify network management and reduce configuration and maintenance workload. - Zero Trust Architecture Support:
• describe: Integrates with Cloudflare Access to support a zero-trust security model with strict authentication of users and devices.
• use: Ensure that only authenticated and authorized users can access internal resources. - Automated traffic management:
• describe: Cloudflare Tunnel automatically handles traffic routing and load balancing.
• use: Improve application reliability and availability and optimize traffic paths.
To summarize it in one sentence: Tunnel is an encrypted tunnel that directly connects the CF edge server and the user device and the network where the device is located. From a technical perspective, it directly connects the CF edge server and the user device installed with the Tunnel client (cloudflared), and can directly access the entire intranet address of the user device with the help of the user device installed with the Tunnel client (assuming that the intranet routing or policy allows it).
For example: Assume that A and B are two devices in the same user network segment, with IP addresses of 192.168.100.1 and 192.168.100.2 respectively. A deploys the web1 website, with the corresponding domain name web1.tangwudi.com and port 80; B deploys the web2 website, with the corresponding domain name web2.tangwudi.com and port 81.
Deploy cloudflared on A. If we want to publish the website web1.tangwudi.com, we only need to configure it as follows:
The website configuration of web2.tangwudi.com on B is as follows. In fact, CF accesses B through the network via cloudflared installed on A, so the address of B must be filled in 192.168.100.2:81, as shown below:
Note 1: See the initiation on Bhttp://192.168.100.2:81The source address of the request is A's address 192.168.100.1.
Note 2: Using the tunnel method is more flexible than using the public IP address to return to the source. For example, you can change the host and some connection parameters:
Note 3: Regardless of the occasion, I always recommend using the Tunnel method when building a website: you don’t have to worry about whether there is a public IP, whether there are legal ports 80 and 443, or even whether it is domestic or foreign.
Note 4: The only disadvantage of using Tunnel is that it is hard to say whether it will be stable during sensitive periods. Of course, it is hard to say whether all the websites accessed by CF will be stable at that time. If CF completely withdraws from the domestic market (even its partners are gone), then the Tunnel method will be completely meaningless.
For detailed configuration steps of Tunnel, please refer to the article:The home data center series uses tunnel technology to allow home broadband without public IP to use cloudflare for free to quickly build a website (recommended).
Afterword
The most difficult "foundation-building" trilogy is finally finished: 3 articles introducing the main functional points and key concepts in the overall CF solution. To be honest, writing this is many times more troublesome than directly writing how to configure a certain function. This is why I have always been reluctant to write the CF series of tutorials: if I don't write it, from the perspective of the overall solution, there is no skeleton, and it will be difficult to extend and refine it later; if I write it, it will cover too many aspects and it will give me a headache (and I guess there are not many people who are really willing to read it~).
Fortunately, I pushed myself this time and finally finished writing it. The follow-up is simple: I only need to describe the configuration of the main functional points. Of course, CF has more functions than these: such as R2 (object storage), CF pages (static page hosting), and other functions that I have not used, but these functions are not node functions in the traffic sequence, and the correlation is not that strong, so I will introduce them in a separate article later.
As for the configuration of the functional modules introduced later, I will not write them in the order of the traffic sequence, because the implementation of certain functions requires the cooperation of some functional modules (for example, complete protection against DDOS attacks requires the cooperation of the DDOS functional module and the WAF functional module), so I will write them in my own logical order.
