Home Data Center Series A bloody murder caused by a "steamed bun": Recording the abnormal blog access phenomenon caused by the upgrade of tailscale in the past few days

A few days ago, I saw that the Tailscale versions running on my devices were all strange. Due to my obsessive-compulsive disorder, I upgraded all the devices running Tailscale (except Qnap's NAS and OpenWRT, because the latest official installation packages of these two are only 1.58) to version 1.78.1:

image.png

没有想到的是,这么多升级设备(包括macos、iphone、ipad和所有linux主机)中,偏偏macos出了问题:正常的升级只是tailscale版本进行升级,并不会影响对应的100开头的固定私有IP地址,而这次升级,3个macos的设备(包括运行博客主站点的m1乞丐版macmini)无一幸免,固定私有IP地址均发生了改变,导致的结果就是在tailscale官网的”Machines”界面里,每一个macos的设备均能看到2条记录(一条是以前的,处于离线状态;一条是新的,处于在线状态)。

I simply deleted the original records without thinking too much, but I forgot the most critical issue: the disaster recovery site I run on the Tencent Cloud host uses the reachability of the previous fixed private IP address of the blog main site macmini as the key indicator of whether the home data center is down (for details, please refer to the article:Home data center series uses cloudflare tunnel to realize automatic takeover of disaster recovery site when WordPress main site fails).

This led to a very serious consequence. Since I upgraded the tailscale version of macmini, the original fixed private IP address has become invalid, causing the detection script of the disaster recovery site of the Tencent Cloud host to always think that the home data center is down, so the disaster recovery site has been enabled. That is to say, during this period of time, there are two connectors in the cloudflare tunnel where the blog domain name is located, while there should be only one connector in the normal state:

image.png

Of course, the same Cloudflare tunnel can also support multiple connectors. This is actually the multi-source load balancing function that comes with Cloudflare tunnel (see the article for details:Cloudflare tutorial series for home data centers (Part 9) Introduction to common Zero Trust functions and multi-scenario usage tutorials), however, there is a big premise: the network structure of multiple source stations is exactly the same. Generally speaking, there is no problem using localhost, which is also the most normal way for cloudflare tunnel to support load balancing of multiple source stations.

但是嘛,由于我的家庭数据中心结构复杂(运行cloudflare tunnel的主机、运行内网waf的主机、运行博客主站点的主机都是独立的),且网络规划肯定和腾讯云轻量服务器的IP地址不一样,所以导致在腾讯云主机上的灾备站点要想在家庭数据中心down掉时接管博客服务(只能接管博客服务,其他的就没办法了,轻量云主机性能只有那么点),需要做一些技术上的”特殊处理”。

具体技术细节我也懒得深究了,总之最后的结果就是对博客的访问要看脸了,有些被分配到家庭数据中心里的博客主站点,有些被分配到腾讯云上的灾备站点,同时因为”特殊处理”的关系,导致cloudflare对源站的探测会间歇性的出问题,经常会出现源站不可访问的502提示:

image.png

而且这个提示还比较随机,访问同一篇文章的时候,有些朋友能遇到,有些又能正常访问,比较像”灵异问题”。

Actually, I have also felt that the blog access is a bit abnormal these days, but because I have adjusted many intranet strategies (including the multi-line DNS function of iQuick) and tinkered with some things (using 2 Apple TVs as backup scientific Internet access outlets and unified tailsacle exit nodes respectively), I have always thought it was because of my tinkering. In addition, I can access it normally with cellular traffic (now it seems that cellular access has better performance), so I didn’t pay much attention to it.

At this point I would like to thank two friends in the group (jdejdndns and RadiantHope) for their reminders:

image.png

image.png

If it weren’t for these two people’s reminder, I wouldn’t be able to confirm that there was really a problem with the blog access. Thank you both here!

In addition, this experience fully verifies a rule of thumb in cutover: Do not make multiple changes in one cutover, otherwise if a problem occurs, you will not know where to start.

The content of the blog is original. Please indicate the source when reprinting! For more blog articles, you can go toSitemapUnderstand. The RSS address of the blog is:https://blog.tangwudi.com/feed, welcome to subscribe; if necessary, you can joinTelegram GroupDiscuss the problem together.
No Comments

Send Comment Edit Comment


				
|´・ω・)ノ
ヾ(≧∇≦*)ゝ
(☆ω☆)
(╯‵□′)╯︵┴─┴
 ̄﹃ ̄
(/ω\)
∠(ᐛ 」∠)_
(๑•̀ㅁ•́ฅ)
→_→
୧(๑•̀⌄•́๑)૭
٩(ˊᗜˋ*)و
(ノ°ο°)ノ
(´இ皿இ`)
⌇●﹏●⌇
(ฅ´ω`ฅ)
(╯°A°)╯︵○○○
φ( ̄∇ ̄o)
ヾ(´・ ・`。)ノ"
( ง ᵒ̌ᵒ̌)ง⁼³₌₃
(ó﹏ò。)
Σ(っ°Д °;)っ
( ,,´・ω・)ノ"(´っω・`。)
╮(╯▽╰)╭
o(*////▽////*)q
>﹏<
( ๑´•ω•) "(ㆆᴗㆆ)
😂
😀
😅
😊
🙂
🙃
😌
😍
😘
😜
😝
😏
😒
🙄
😳
😡
😔
😫
😱
😭
💩
👻
🙌
🖕
👍
👫
👬
👭
🌚
🌝
🙈
💊
😶
🙏
🍦
🍉
😣
Source: github.com/k4yt3x/flowerhd
Emoticons
Emoji
Little Dinosaur
flower!
Previous
Next
       

This site has disabled the right mouse button and various shortcut keys. The code block content can be copied directly by clicking the copy button in the upper right corner

en_US