Preface
I officially started playing with routers probably with Netgear's R7000:

However, back then it seemed to still be the era of 8M ADSL, with an uplink of only 512k, so I didn't tinker with it much at first and just used the official firmware. Moreover, in my memory, the blocking back then was far from comparable to what it is now, and the impact wasn't that significant yet, so I didn't have the idea of spending time to mess around with it.
A few years later, the censorship became stricter and stricter, which started to affect my daily work (as a technical engineer in a foreign company, I needed to access foreign websites too often). I had to consider the circumvention issue, so I flashed the Merlin system onto the R7000 and installed Shadowsocks using the software center to make do with it (to be fair, the performance of the R7000 is still decent even today, just that the Wi-Fi is a bit weak).
The tragedy caused by multi-dialing - starting to tinker
One day, I suddenly saw an article about multi-dialing. Holy crap, you can actually do that? I saw that Merlin also supported dual-line single-dialing, so I tried it immediately. I was lucky to find that my home Telecom broadband actually supported it. However, Merlin's dual-line dual-dialing feature was too simple—either active-standby or just load balancing both lines together. The playability was too poor (after all, I professionally sell link load balancing equipment, so having high standards is quite reasonable, right?~). So I considered other solutions that support multi-dialing, and then I came across LEDE and iKuai.
LEDE (which is a branch of OpenWRT) is actually fine, but for one thing, I'm not very used to LEDE's configuration logic, such as iptables concepts like the input chain, output chain, and forward chain (compared to the habits I formed after configuring a large number of Cisco and Huawei Layer 3 switches, routers, and various models of firewalls). I couldn't react instantly and needed my brain to translate it, which is very awkward for someone like me who relies on instinct for configuration and troubleshooting. Secondly, after trying iKuai, I found that besides its perfect support for multi-dialing, its various other features are also tightly integrated with the multi-dialing function (see my other article:iKuai Soft Router Series - The Strongest Multi-Dialing Soft Router: iKuai), and it also fits the configuration habits of Chinese users very well. So in the end, I chose iKuai as the primary router and the R7000 as the secondary router. Thus, the first network structure change became like this:

In this structure, I directly pointed the default gateway of all devices to the LAN port address of the R7000, while the WAN port of the R7000 used static IP mode, pointing its default route directly to the LAN port address of iKuai. Additionally, the NAT function of the R7000 can be turned off here:

This is because they are all daisy-chained in the network, acting merely as a Layer 3 switch, so NAT is not needed at all. Furthermore, turning off NAT has another benefit: when iKuai does port forwarding, it can point directly to the real intranet address of the hosts in the 192.168.1.x subnet (of course, a return route for the 192.168.1.x subnet must be written on iKuai pointing to Merlin's WAN port address); otherwise, a secondary forwarding would be required on the R7000. Double NAT from inside out, and double forwarding from outside in—it's redundant and offers no benefits, so turning off NAT directly is the way to go.
Some of you might have a question here: why didn't I use the bypass gateway mode for the R7000? Actually, from a professional perspective, there is no such concept as a "bypass router". The correct term should be supporting single-IP promiscuous mode packet receiving, forwarding, and performing NAT simultaneously.
Generally speaking, a normal host device's network card in normal mode will only receive packets whose destination address is the network card's own IP, and other packets will be directly discarded. To receive packets whose destination IP is not its own, the network card needs to be set to promiscuous mode (I remember being prompted about this when installing Wireshark on Windows in the past).
So, to achieve the function of a bypass gateway, the device must support at least two functions:
The network card must support promiscuous mode or be able to be set to promiscuous mode
In this regard, the LAN ports of all routers are enabled by default, whereas Linux and similar devices can enable them using commands, for example:
ip link set eth0 promisc on
Support single-IP forwarding while simultaneously performing NAT
In this regard, consumer-grade broadband routers fall short, because they generally can only receive packets from the LAN port and send them out through the WAN port while performing NAT using the WAN port's IP address (better systems have a toggle to enable/disable NAT, but on average ones, NAT is hardcoded and cannot be disabled).
For Linux, however, this is not an issue at all. After all, the FORWARD chain in iptables is designed exactly for this. If a Linux machine has two network cards with IPs in different subnets and forwarding is enabled, it instantly transforms into a Layer 3 switch (of course, switching performance is another story). If there is only one network card, enabling forwarding and then IP masquerading (which is NAT) directly turns it into what is now commonly called a "one-arm bypass router".
The first optimization
After using this setup for a few years, there weren't actually any major issues, except that I frequently tinkered with the R7000—flashing Merlin firmware, upgrading Shadowsocks, and rebooting the R7000 whenever things acted up. These reboots caused network interruptions, which was very annoying. Additionally, I wanted the main router and all clients to be on the same subnet to facilitate operations like Wake-on-LAN. So, after some thought, I decided to change the network topology. Since the R7000 was already an older model by then, with poor support for Merlin firmware and proxy plugins, I bit the bullet and upgraded to the ASUS RT-AC66U B1:

After the second network topology change, it looked like this:

The advantage of this setup is that the default gateway for all clients on the main router simply needs to point to the LAN IP of iKuai. As for the users who need bypass proxying, you only need to configure port-based or domain-based policy routing on iKuai:


Simply direct the traffic from the corresponding source IPs or specific domains to iKuai's WAN3 port. Meanwhile, even if the AC66U B1 crashes or the proxy service fails, it won't affect normal internet access for other clients (similarly, you can disable NAT on WAN3 in iKuai, though this requires adding a static return route to the local network on the AC66U B1).
However, this topology has its drawbacks. One of the PPPoE dial-ups is handled by the AC66U B1, making O&M and load balancing less convenient than having all three dial-ups managed centrally on iKuai. Furthermore, if you want to configure port forwarding for WAN3, you have to do it on the AC66U B1, which is somewhat inconvenient. Overall, though, the pros still outweigh the cons.
Second optimization
A few years later, I found the wiring in the diagram above increasingly awkward—my OCD was acting up, and I wanted to optimize it further. So, I took the opportunity to upgrade the router to my current RT-AC86U:

After the third network topology adjustment, it became the following:

The advantage of this setup is that all PPPoE dial-ups are consolidated back onto iKuai, allowing the uplink bandwidth of all three lines to be easily utilized. Meanwhile, clients still only need to point their default gateway to iKuai. iKuai then routes the proxy-bound traffic from WAN3 to the LAN port of the AC86U based on the source IP or destination domain (NAT must be enabled; by default, iKuai enables NAT on all WAN ports). The WAN port of the AC86U is then connected back to the local network (which also requires NAT enabled). This deployment method preserves the benefit that an AC86U crash or proxy failure won't disrupt normal internet access for other clients. This is the ultimate topology for a dual-hardware router setup.
P.S.: Since WAN1 handles triple-dialing over a single line, it freed up WAN2 to connect to IPTV. So, I went ahead and moved the IPTV functionality from Merlin to iKuai. This completely satisfied my OCD. Crucially, it further streamlined the Merlin router's role, making it much easier to replace in the future. For configuring IPTV on iKuai, please refer to the article:iKuai Soft Router Series: Achieving Telecom IPTV All-Media Playback。
Final optimization
However, I still find this setup a bit annoying: First, a large part of the Asus router's expensive price is for its Wi-Fi, but I use Netgear's mesh for my Wi-Fi, so the Asus router's Wi-Fi is a complete waste for me; Second, the heavy traffic of normal internet access now does not go through the AC86u, and the router's main purpose is bypassing the firewall, so the performance requirements are actually not that high. In this case, running any virtualized device is actually perfectly fine. Keeping an extra piece of hardware running 365 days a year just for bypassing the firewall is really not cost-effective when considering electricity bills; Third, what if it breaks? Am I supposed to buy another one? Using it before was due to historical reasons, which was understandable, but buying another one would be too lame. That's why I started messing around with OpenWrt (see my previous two articles:OpenWrt Soft Router Series: Detailed Tutorial on Deploying OpenWrt (23.05.2) on PVE和OpenWrt Soft Router Series - 3 Must-Install Essential Tools: openclash+sftpserver+tailscale Installation Tutorial)。
And then came the final solution:

In fact, it is just using a virtual machine on PVE to run OpenWRT to replace the AC86u. Of course, OpenWRT needs to be configured first, with the following key points:
OpenWRT hardware needs dual network cards (only one by default, one needs to be added):

Correctly configure interfaces on OpenWRT


Correctly configure firewall zones
The LAN zone and WAN zone here are slightly different from the usual case, because normally traffic only enters from the LAN port, exits from the WAN port, and undergoes NAT. In my structure, traffic enters both the LAN and WAN. The difference is that the LAN port is dedicated to iKuai, while the WAN port is for intranet users (not for configuring the default gateway, but for intranet users to access the built-in socks5 and HTTP proxies of OpenClash).
For iptables, it only looks at the configuration of the input, output, and forward chains corresponding to the network card zone. Under my structure, the firewall zone settings are as follows:

Two ways to set the default gateway on the client side
User default gateway points to the LAN port IP of OpenWRT (not recommended)
For intranet users, if the default gateway is pointed directly to the WAN port IP of OpenWRT, all traffic not in the local network segment will directly reach OpenWRT (including traffic that needs scientific internet access). OpenWRT will directly process the traffic that needs scientific internet access according to existing policies, and the traffic that does not need scientific internet access will be directly NATed by OpenWRT and sent to the LAN port of iKuai for normal internet traffic processing. This is what is commonly known as the single-arm bypass router mode. This deployment method is simple and crude, with few benefits but a lot of disadvantages: once OpenWRT crashes, all clients pointing to this gateway will lose internet access; part of OpenWRT's performance is wasted on forwarding and NATing traffic that does not need scientific internet access, while the traffic that actually needs scientific internet access consumes very little performance; if port forwarding is configured on the main iKuai router to a client, and the client's default gateway points to OpenWRT, it may cause OpenWRT to send ICMP redirects to the client, etc.
User default gateway points to the LAN port IP of iKuai (strongly recommended)
If the default gateway of intranet users points to the LAN port address of iKuai, iKuai can send user requests that need scientific internet access through the WAN3 port to the LAN port of OpenWRT according to the traffic splitting policy. Then, OpenWRT processes the scientific internet access traffic according to existing policies. Although this adds the extra step of iKuai traffic splitting, it brings huge benefits such as clients not needing to change their gateways, scientific internet access device failures not affecting normal internet access, and facilitating one-step scientific internet access for new Linux systems (think about the embarrassing situations of a newly installed Linux system where apt cannot download software; having to modify configuration files to change IPs but having no vim, and using vi makes you want to throw up; wanting SSH access but being unable to download openssh-server, etc. With iKuai's intervention, you only need to throw this new system's IP to OpenWRT in the traffic splitting settings to solve it). Any normal person would choose the latter.
When I wanted to cut over from the AC86u to OpenWRT, I didn't need to change any configuration. I just needed to unplug the network cable that originally connected the iKuai WAN3 port directly to the AC86u LAN port and plug it into the local area network switch (provided that the IP address of the OpenWRT LAN port is the same as the LAN port address of the AC86u).
Note: Actually, it is not absolutely necessary to use iKuai as the main router to do this. Any router that supports multiple WAN ports and policy routing—such as routing based on source IP address or supporting routing based on domain name—will work. It's just that I haven't seen anything that can compare to iKuai in terms of traffic splitting and routing; iKuai is just too powerful in this area.
Summary
In fact, the most important thing about the final deployment scheme is the concept. It doesn't necessarily have to be hardware or a virtual machine (for example, you can use a virtual machine to install OpenWRT inside an independently deployed hardware iKuai; or install both iKuai and OpenWRT on ESXi; or like me, iKuai is hardware and OpenWRT is deployed as a virtual machine, etc.). The most critical part is the connection logic between iKuai and OpenWRT (which can be a physical connection or a virtual connection through a vSwitch). You can flexibly choose the scheme that suits you based on your actual environment.
Hello, I deployed it according to your scheme, and normal internet access is fine, but I also want to install tailscale on openwrt. After installing it, I found that the node cannot connect to other devices in the intranet.
I installed tailscale on openwrt, and it works fine for me. I don't quite understand what you mean by ”the node cannot connect to other devices in the intranet”. Does this ”cannot connect” mean that openwrt cannot actively ping other devices in the LAN, or that other devices in the LAN cannot ping openwrt? Nodes with tailscale installed can use the ”tailscale” command in the terminal to configure some functions, such as whether to allow LAN connections, you can take a look.
My devices are all connected to the tailscale network, with the subnet 100.64.1.0/24. Logically, after openwrt joins tailscale, with these two parameters `–accept-routes=true –advertise-routes 192.168.100.0/24`, devices accessing the internet through openwrt should also be able to join the tailscale network. However, my other devices can only access 100.64.1.10 (openwrt) and cannot access the 192.168.100.0/24 subnet. The LAN subnet of ikuai is 192.168.1.1/24, the LAN port of openwrt is 192.168.100.0/24, and PVE is 192.168.100.10. ikuai cannot even ping 100.64.1.1 (headscale and derp server), while PVE and other devices can ping 100.64.1.1, but 100.64.1.1 cannot ping 192.168.100.1, 192.168.100.10.
Without a topology diagram, it's really hard to visualize just by looking at the IP address ranges, but I guess you mean that other devices with tailscale deployed cannot access the 192.168.100.0/24 subnet where openwrt's LAN port is located? I haven't actually done this kind of subnet routing using tailscale, because all devices that need to access each other have tailscale installed, so there was no such need. However, I think a key point is whether other devices with tailscale deployed can see the route to the 192.168.100.0/24 subnet, and the gateway should point to 100.64.1.10 (openwrt). Logically, if your settings are normal and advertise-routes is correctly set, other devices with tailscale deployed should be able to see it, which is the basis for communicating with the 192.168.100.10/24 subnet. Also: is `–accept-routes=true` set on other tailscale devices?
When I deployed it before, I only needed to add `–accept-routes` on openwrt. Do other tailscale devices also need to enable this parameter? I'll go back and try it.
“–accept-routes”, I think this switch should control whether to accept routes advertised by other tailscale nodes.
Hello boss, I ran into another problem... My current IPv6 is very strange. It is normal within the LAN, and can be pinged from the internet, but I cannot access port services via IPv6. It seems like a firewall issue, but I have turned off all firewalls on openwrt, and enabled ikuai's as well. I didn't see any firewall settings on the optical modem either, but it still doesn't work.
When encountering this kind of problem, you need to capture packets on ikuai to confirm whether the packets pinged from the external network actually reach ikuai's WAN port. If they reach it but cannot be received inside, you can confirm it is an IPv6 firewall issue.
I want to ask, eth0 and eth1 are no longer present in the new version of openwrt, there is only br-lan, as well as the Ethernet adapter port wan, and switch interface eth0. How should I set it up? Is it the last scheme? I guess your openwrt kernel version is below 5.5?
Just add a network card in the virtual machine, and then set it up in openwrt. By default, there is only one eth0, and any extra ones must have been added by yourself. As for the switch, you don't need to configure anything. Multiple subnets can run in one broadcast domain, but generally, this is not recommended because broadcast packets in a production environment will affect each other, but you don't have to worry about it in a network you control yourself.
Thanks to the blogger, you gave a great idea to solve the problem of unifying the configuration of one-arm routing.
And a small suggestion: you can install openwrt inside ikuai's virtual machine, and then combine it with ikuai's port flow control next-hop policy to achieve a more natural-looking collaboration.
2 benefits:
Then I have a small suggestion: you can install openwrt inside iKuai's virtual machine, and then combine it with iKuai's port flow division next-hop policy to achieve a more natural-looking collaboration.
2 benefits:
1. openwrt is installed inside iKuai, the iKuai device is physically isolated from the nas, so future nas problems or self-reinstallation/tinkering won't affect scientific internet access;
2. iKuai's cross-three-layer application combined with openwrt's SNMPD service can achieve more detailed terminal traffic statistics.
I also installed virtual machines in iKuai before, but later I specially bought an Intel CPU mini host to install PVE, and then migrated all virtual machines and Docker there, which makes O&M management easier.
Now I only let my iKuai focus on being an egress device (it's an industrial PC bought separately, not in the NAS). Firstly, because when running virtual machines before, the CPU fan of iKuai annoyed me, and secondly, because my iKuai has 3 dial-ups, and each link has its own role. In terms of importance, it is much greater than openwrt. If they were really installed together, I would worry that openwrt might affect my iKuai~~. Besides, openwrt is actually just my backup device. I currently mainly use the AC86u. I set up an openwrt because scientific internet access was unstable for a while, and I couldn't determine whether it was an issue with the AC86u or the scientific internet access itself, so I installed an openwrt as a backup testing method.
Now I only let my iKuai focus on being an egress device (it's an industrial PC bought separately, not in the NAS). Firstly, because when running virtual machines before, the CPU fan of iKuai annoyed me, and secondly, because my iKuai has 3 dial-ups, and each link has its own role. In terms of importance, it is much greater than openwrt. If they were really installed together, I would worry that openwrt might affect my iKuai~~. Besides, openwrt is actually just my backup device. I currently mainly use the AC86u. I set up an openwrt because scientific internet access was unstable for a while, and I couldn't determine whether it was an issue with the AC86u or the scientific internet access itself, so I installed an openwrt as a backup testing method.
Just wanted to ask, if a port flow division policy is added, and openwrt goes down, will it affect the normal use of the redirected devices?
It depends on whether you have checked the ”Line Binding” option. However, I feel that the judgment logic of this option is likely to be quite rough, perhaps relying on whether the upper gateway is reachable or whether the line interface is up or down (after all, there is no specific option, I can only judge by conventional logic). Depending on the different judgment logic and how your openwrt goes down (for example, completely down, where even pinging the interface address fails, or half-dead, where pinging the interface works but it actually doesn't work anymore), the final actual effect may not be what you want. So this is also one of the reasons why I separated iKuai and openwrt. Regardless of whether openwrt is alive or dead, the iKuai main router must not be affected at all. After all, in my ”home data center” solution, the stable and normal operation of iKuai is of paramount importance.