Background of the project

I have been thinking about how to "use the free tunnel of Cloudflare's free plan to achieve multi-node WordPress local access and disaster recovery solutions" these days. However, there is a big prerequisite for implementing this solution: the content of the WordPress sites on these nodes is the same. In other words, the data of the multi-node WordPress must be synchronized.

Looking back at my current data synchronization solution for multiple WordPress sites, it still relies on the WPvivid backup plugin on the main WordPress site to periodically generate a complete backup of the entire WordPress site (program files + database), and then uses SyncThing to synchronize it in real time to the location of the WPvivid backup plugin on other WordPress sites. Finally, when needed, it is manually synchronized using a "one-click restore" method. ...Isn't that too inefficient? And I don't even have a plan for scheduled restores (mainly because I'm lazy, and it depends on my mood at the time).

All in all, this simple synchronization solution is no longer suitable for my currentidentityThe scene is in urgent need of upgrading and renovation.

Thinking about database synchronization solution

With Syncthing as one of my underlying supporting technologies, the synchronization of WordPress program data itself is no longer a problem. The key lies in what solution should be used for the synchronization of database content corresponding to multi-node WordPress. Generally speaking, there are three options:
1. Directly use the same mariadb database

If the purpose of multiple sites is just redundancy: by default, blog access requests only access the primary site, and only when the primary site fails will the backup site be switched, and only when the backup site fails will the disaster recovery site be switched. Then there is no problem with using one database, because there is only one WordPress site reading and writing at the same time. However, in this case, if there is a problem with the only database, the whole thing will collapse, and if my home data center loses network or power, the database will also be lost. A single disaster recovery site with only one WordPress is useless without a database, so I just pass it.
2. Use database master-slave synchronization

Considering that both the main WordPress site and the backup site are in the home data center, you can consider connecting both the main and backup WordPress sites to the same mariadb database in the home data center. The disaster recovery site runs a separate mariadb database. The mariadb in the home data center is the master and the mariadb in the disaster recovery site is the slave. Configure master-slave synchronization. There will definitely be no problem (tailscale, as one of the underlying supporting technologies, has made an indelible contribution).

However, the problem with this approach is that the database inside the home data center still has a single point of failure; and the master-slave synchronization across the wide area network may cause master-slave replication failure if the delay is high; in addition, I only have data access once every few days or longer, and this kind of synchronization feels like using a cannon to kill a mosquito (and it is synchronized over a wide area network~), so I passed it.
3. Each WordPress site runs a mariadb database on the local machine

This method is actually the current method, which forms database redundancy inside the home data center: the main site and the backup site are deployed on macmini and intermini respectively, and each node has wordpress and mariadb databases, and the wordpress of each node reads and writes the local mariadb database. wordpress realizes primary and backup redundancy through load balancing, and the two mariadb databases are independent (of course, mariadb database can also be load balanced, but I am too lazy to bother with it, so it doesn't make much sense). At the same time, wordpress and mariadb databases are also deployed at the disaster recovery site, so that when problems occur in the home data center, the normal operation of wordpress at the disaster recovery site will not be affected (the disaster recovery site accesses the local database, which has nothing to do with the database in the home data center). The only problem is how to ensure the instant update of all node databases (I didn't think of a good method before, so I could only rely on the WPvivid backup plug-in).

After having chevereto's multi-point timing synchronization solution (see article:Tutorial on how to synchronize multiple nodes of Chevereto image bed in home data center series), the last pit has been filled, the synchronization of wordpress can also use the same idea: on the wordpress main site, use syncthing to complete the synchronization of wordpress program data to other nodes and the synchronization of the exported wordpress.sql file, and then other wordpress nodes can complete the import of the wordpress.sql file on their own mariadb databases (in theory, no matter how many nodes there are, the same method can be used).

The problem of timely updates is actually very easy to solve. I’ll keep you in suspense and talk about it later. But before that, I need to re-plan the wordpress folder to be synchronized in the syncthing solution.

Deploy Syncthing to synchronize wordpress folders

Previously, the folder synchronized by WordPress in Syncthing was only the folder of WPvivid backup plug-in (mainly to synchronize the backup files saved to local by WPvivid plug-in of WordPress main site to the WPvivid plug-in directory of other nodes, so as to facilitate one-key recovery when needed). Now, we need to change it to synchronize the main WordPress site using-vThe parameter is mounted to the entire "html" directory of Docker:

For the configuration details of syncthing, please refer to the previous article:Docker series: A detailed tutorial on how to synchronize multiple folders using Docker based on syncthingI will not repeat it here. For subsequent experiments, SyncThing added an aliyun node this time. The docker run command format is as follows:

docker run --name syncthing-aliyun -d --restart=always --net=public-net \ --hostname=syncthing-aliyun \ -e TZ=Asia/Shanghai \ -p 8384:8384 \ -p 22000:22000 /tcp \ -p 22000:22000/udp \ -p 21027:21027/udp \ -v /docker/syncthing/config:/config \ -v /docker/wordpress:/data1 \ lscr.io/linuxserver/syncthing:latest

Use the tailscale address to add the node on the synchronization source syncthing-macmini, and complete the synchronization settings for the wordpress folder on all nodes.
1. Effect on syncthing-macmini:

2. Effect on syncthing-intermini:

3. Effect on syncthing-tc:

4. Effect on syncthing-aliyun:

As long as this state is reached, the subsequent synchronization of syncthing will basically not have any problems, and the most troublesome step is finally completed.

Four points to note:
1. Map to the directory inside the container, such as the one in the command above/docker/wordpress, be sure to change the permissions of the wordpress folder to 777 on the host machine of each node, otherwise synchronization will fail.
2. If this error occurs:

You can solve this by manually creating a folder named ".stfolder" within your WordPress folder:

3. If the status of the WordPress folder on other nodes is not "Sync Complete":

Instead, it shows a red "out of sync" error. Let's first find the reason for the failure:


The only workable steps were to change permissions, remove folders from the syncthing nodes that had lost synchronization, delete the contents of the folders, restart the syncthing container, and then add a ".stfolder" folder and re-share it from the host. This process was repeated until all states were normal.
4. It is recommended that, except for the main WordPress node, all other nodes should have their WordPress folders cleared (if there were any files originally), and then a new ".stfolder" folder created before starting to add shared folders from the main node and completing the synchronization.

Note: syncthing may have other errors. You can search online. There are many articles about it. I certainly cannot cover all the problems here.

Deploy and initialize mariadb

Then there's the database issue, because the "wp-config.php" configuration file is the same across all WordPress nodes, for example:

Therefore, we need to ensure that the way WordPress accesses the mariadb database on each node is the same (in this article, they can all be accessed through mariadb01). The normal way is to place the mariadb database docker and the wordpress docker in the same non-default bridge when creating a new database, as shown in the command in syncthing of the new Alibaba Cloud node above.--net=public-netSimilarly, you need to ensure that the Docker name for MariaDB on all nodes is the same as the one set in "wp-config.php". In my case, it's all MariaDB01.

The creation command of mariadb01 is as follows:

docker run --name=mariadb01 -d --restart=always --net=public-net \ -p 3306:3306 \ -v /docker/mariadb/db:/var/lib/mysql \ -e MARIADB_ROOT_PASSWORD=yourpassword \ mariadb:10.11

After creating the mariadb database for each node, you still need to initialize it: create a WordPress library and the account and password corresponding to the library, and grant access rights to the WordPress library to the account (for related operations on initializing the library, see the article:Tips and tricks: Create a new empty database and grant permissions to corresponding usersOf course, you can also use the method you are familiar with to initialize, such as command line, phpmyadmin, etc.).

MariaDB database synchronization

Export the main site WordPress database

After having experience with Chevereto database synchronization, I still use the same method: export the wordpress library in the mariadb database on the main wordpress site to the newly created DB directory in the wordpress synchronization folder configured by syncthing:

Create a new script named "export_wordpress_database.sh" to perform the export. The script content is as follows:

#!/bin/sh # Remote ssh login will not load environment variables by default. Actively load it to prepare for remote ssh to execute this script. Local scripts do not need source ~/.bash_profile # Execute the export wordpress library command to export to wordpress/db that has been configured with syncthing synchronizationIn the folder, docker exec -u root mariaDB01 mysqldump -uroot -pyourpassword --databases wordpress > /Volumes/data/docker/wordpress/db/wordpress.sql # Run to complete echo 'end'

Don't forget to give execute permissions to the script:

chmod +x export_wordpress_database.sh # needs to enter the correct path to execute

Then place the scripts in any path. For example, I created a "startup" directory under my macOS user directory. I put all scripts that need to be run remotely via SSH in this directory for easy management, as shown in the image below:

Now you only need to run this script to generate a full backup file wordpress.sql of the wordpress library on the main wordpress site in the wordpress/db directory. As long as the previous syncthing synchronization directory is configured normally, the wordpress.sql file can be quickly synchronized to the DB directory of all other wordpress sites:

Main wordpress site (macmini):

Alternative wordpress site (intermini):

Disaster recovery WordPress site (Tencent Cloud Server):

Note: Syncthing has its own default interval scanning time (3600 seconds). The time to complete data synchronization varies with the size of the file to be synchronized and the connection rate of each Syncthing node. My WordPress database is tens of megabytes, and it takes at most 1-2 minutes for all nodes to complete synchronization. However, this is the time after the scan finds the changes. As for when to start scanning, it depends on luck. I can only guarantee that it will be scanned within 3600 seconds.

Import the wordpress library in mariadb of other nodes

We still need to rely on shell scripts to accomplish this. Create a new script named "wordpress_import.sh" in the MariaDB mount folder on all WordPress nodes except the main WordPress site, with the following content:

#!/bin/sh mysql -uroot -pp@ssw0rd < /var/lib/mysql/wordpress.sql # Run completed echo 'end'

This script needs to be run in the mariadb container./var/lib/mysql/Import the wordpress.sql file synchronized from the main wordpress site into the local mariadb.

Note 1: Why<The left side is empty? Because the mysqldump command was used earlier--database wordpressThe parameter exported wordpress.sql file contains commands such as create wordpress, as follows:

So you don't need to specify it. Also don't forget to give it execution permissions:

chmod +x wordpress_import.sh # needs to enter the correct path to execute

Note 2: Why create the script "wordpress_import.sh" in the MariaDB mount folder? Because it's not possible to directly import files from the host machine's file system into the MariaDB container, so an alternative is needed.

Then create a new shell script "import_wordpress_database.sh" with the following content:

#!/bin/sh # Remote ssh login will not load environment variables by default. Actively load it. This is to prepare for remote ssh to execute this script. Local script running does not require source ~/.bashrc # Note that the file names of environment variables on different systems are different. For example: macos is bash_profile, debian is bashrc, please modify according to your actual environment # Delete the wordpress library directly. It is too troublesome to delete the table, and I am afraid that a new table will be added due to a certain version change. However, it seems that it is not necessary to delete it. docker exec -u root mariadb01 mysql -uroot -pyourpassword -e "drop database if exists wordpress;" # Copy the synchronized wordpress.sql to the working directory of mariadb. This is the workaround mentioned above cp /docker/wordpress/db/wordpress.sql /docker/mariadb/db/01/wordpress.sql # Import the wordpress.sql exported from the main wordpress site into the local mariadb database docker exec -u root mariadb01 bash /var/lib/mysql/wordpress_import.sh # Run completed echo 'end'

Put the script in the path you are used to. I put it directly in the /root/script directory.

After completing the above operations, just run/root/import_wordpress_database.shThis script can accomplish this:
Step 1: Delete the mariadb database of this nodewordpressLibrary (it seems that this step can be omitted, but it is cleaner this way);
Step 2: Copy the latest wordpress.sql file in /docker/wordpress/db to the mount directory of the mariadb container
Step 3: Import wordpress.sql into the mariadb container.

Deploy WordPress on other nodes

After the previous steps, the WordPress program files and databases on other WordPress nodes except the main WordPress node have been synchronized. If these nodes have already deployed WordPress nodes, they can be accessed directly. Because my Alibaba Cloud node is newly added, I need to deploy the WordPress container. I only need to use the following command:

docker run --name=wordpress -d --restart=always --net=public-net \ -p 80:80 \ -v /docker/wordpress/html:/var/www/html \ # Pay attention to the mounted wordpress folder path and do not travel wordpress

Then you can access it directly.

Note: By default, WordPress limits the access domain name. If you want to remove the access restriction, please refer to my other article:A simple tutorial on how to set up WordPress to support multiple domain name access.

Use inotify to monitor folder changes and execute scripts

Process Logic

Actually, the preceding steps were all manual operations, which only qualify as a multi-node WordPress synchronization solution, and have nothing to do with the "near" real-time synchronization solution I mentioned in the title. Indeed, so the next step is the key: inotify.

Inotify is a Linux tool for monitoring folder changes. It can monitor changes to folder contents and then execute specified scripts. This relates to our previous operation: when I run the script "export_wordpress_database.sh" to export the WordPress database on the main WordPress site, a new `wordpress.sql` script is generated in the "wordpress/db" directory of the main site. The new `wordpress.sql` script is then detected by Syncthing and synchronized to the `db` directory on all other WordPress nodes. Inotify running on other WordPress nodes monitors the "wordpress/db" folder on the local node. Once a change is detected, it executes the local "import_wordpress_database.sh" script to import the WordPress database. Then, WordPress synchronization is complete on all nodes—perfect!

Install and deploy inotify

apt update
apt install inotify-tools

After the installation is complete, you can use the two commands inotifywait and inotifywatch. Among them, inotifywait is what we want to use to monitor changes in the wordpress/db folder, and inotifywatch is to count the number of file system accesses, which is not used here.

We need to write a script.”monitor.sh:, the content is as follows:

#!/bin/bash
watchdir="/docker/wordpress/db"  # defines the content that needs to be monitored. I will monitor the db directory here.
script="/root/import_wordpress_database.sh" # defines the script to be executed
while 
inotifywait -e modify $watchdir; # -e modify dir is for monitoringCheck if any files in the dir directory have been modified -o modifylog writes the monitored events (in this case, the file being modified) to...ModifyLog
do
timestamp=$(date)
sleep 1m # Wait for 1 minute, because it takes some time to synchronize files through syncthing. If you don't wait, file changes will be recorded all the time.
bash $script # executes the script to import the wordpress.sql file
done

Grant execute permissions:

chmod +x monitor.sh

Then add monitor.sh to the startup (see my other article:3 common ways to set up commands or scripts for Debian series to start at boot).
Finally, make sure monitor.sh is running properly on all other wordpress nodes.:

Verify the actual effect of the solution

1. Execute the script "export_wordpress_database.sh" on the main WordPress node.”

Go to the export directory to confirm:

2. Verify whether syncthing is synchronized successfully

Since the default syncthing rescan time is 1 hour, I don’t want to wait that long, so I click rescan (note that the time I clicked is 08:06):

---

What did I do between the export time 08:01 and 08:06? Because my little bear egg cooker beeped to remind me that the eggs were cooked, I went to have breakfast first.

In fact, if syncthing runs normally at this time, the synchronization is basically completed instantly (mainly because the wordpress.sql file is not large, only 38 MB). In the terminal window of initofywait, you can see the folder modify event information. Take the Alibaba Cloud node as an example:

---

Then we go to other nodes in turn to confirm, first confirming whether the syncthing synchronization is successful.
Alternate site:

Tencent disaster recovery site:

Alibaba disaster recovery site:

It can be seen that the synchronization of syncthing was successful.

Finally, let's confirm whether inotifywait has successfully monitored/docker/wordpress/dbThe wordpress.sql file in the folder has been modified, and the script "import_wordpress_database.sh" has been executed. If the script is executed as follows:

Then we look at several nodes in turn/docker/mariadb/db/01/wordpress.sqlThe modification time of the file (actually you should look at the last commanddocker exec -u root mariadb01 bash /var/lib/mysql/wordpress_import.shHowever, I can’t think of any good way to prove this result, and it doesn’t seem authoritative enough, so I’ll look at the generation time of wordpress.sql in the previous step).
Standby Node:

Tencent Node:

Alibaba Node:

Basically, it was one minute after I clicked the "Rescan" button in the syncthing WordPress folder (08:06), which proves that inotifywait can indeed monitor./docker/wordpress/dbThe folder changes and executes the specified script:
import_wordpress_database.sh, added mine:WordPress multi-node "semi-automatic" and "nearly" real-time synchronization solutionThe last piece of the puzzle.

Note 1: "Semi-automatic" means that after updating the main WordPress site, I need to manually run a database export script, "export_wordpress_database.sh," while the rest is fully automatic. Frankly, "semi-automatic" is a bit modest; it should be "0.9 automatic," but it's still far from "fully automatic." Achieving full automation isn't impossible, but it requires further in-depth research into WordPress's working principles, mainly how to determine if WordPress has updated content (posts, statuses, pages), etc., because plugins and WordPress versions can update automatically, so these cannot be used as a basis for judgment (actually, you could start by checking for new content in the database—wow, that's exciting!).

I started having second thoughts about this, and I really didn't want to delve any deeper into it. So I decided to just manually run a script to output the wordpress.sql file to a specified folder to trigger the subsequent process. Actually, this goes against my principle of "if there are difficulties, we should tackle them; if there are no difficulties, we should create them." But it's just too tiring and a waste of my brain cells. I'll continue researching it when I'm in the mood later.

Note: After using it for a while, it is still not possible to directly synchronize the entire HTML directory of WordPress. This is mainly because due to the main node version upgrade or plugin update, directly synchronizing the entire HTML directory to other nodes often causes a "fatal error" in WordPress. The best way is to only synchronize the directory containing the wordpress.sql file. This way, you can simply export, synchronize, and import the WordPress database, which will solve the problem.

Note 2: The term "near-real-time" synchronization refers to the fact that Syncthing performs a full scan every 1 hour by default (this is somewhat luck-based; sometimes it scans quickly, and sometimes it takes a long time to respond, so I manually clicked the "rescan" button during testing). 1 hour is perfectly acceptable for my needs, as it's just a personal blog and not some high-standard enterprise application, so I've been too lazy to change it. If you really need higher standards, just reduce the "full scan interval" in Syncthing.

Summarize

Actually, if you just want to synchronize published posts across multiple WordPress sites, there are simpler methods: for example, using plugins. There are more than one such plugin, and although they have some settings and their effectiveness has its flaws, they are still much simpler than my method. Furthermore, the method I used before—"based on the WPvivid backup plugin for scheduled backups, combined with Syncthing synchronization, plus manual restoration when needed" (such a long description still feels rather rudimentary)—is not entirely unusable.

But for me, by constantly trying, constantly learning new technologies (for example, this article forced me to study shell scripts), and constantly optimizing solutions, even if it is just a little improvement each time, and learning a little new knowledge, but after accumulating day by day, maybe one day I will become a master of magic! (Sunflower in hand, hahahaha, I have the country, hahahaha, it sounds familiar, right?)

Of course, as mentioned at the beginning of the article, this article can only be considered a preliminary technical study. The final research will be on "implementing a multi-node WordPress local access and disaster recovery solution using the Cloudflare free program's tunnel." I don't know if it will work, but let's study it and see.