Preface

The previously built umima (see article:Docker series builds a website traffic monitoring system based on umami) version is 1.33.1:

The latest official version of GitHub is 2.11.3:

For someone who has an obsessive-compulsive disorder about upgrading, how can I bear this? ? So I started to upgrade, but after a long time, the version didn't change (later I found out that the mirror address was invalid), and I also found that umami suddenly couldn't receive data after a while (I think it should be when I re-deployed umami, the website-id changed, but it didn't change on the dashboard), which was embarrassing.

However, I thought, I might as well take this opportunity to review the installation method of umami. After all, when I built umami before, I didn’t know anything at the time. I followed other people’s tutorials step by step and didn’t dare to make any mistakes. Now, after a period of training, I have improved a little. In addition, it feels like the umami tutorials on the Internet are all from a long time ago, so it’s time to update them. So there is the journey of this article.

But still based on the old saying: "If there are difficulties, you must go ahead; if there are no difficulties, you must create difficulties and go ahead", I chose the more rugged path this time: using the source code method to build umami.

Building umami

Use git to build umami from source code (not recommended, troublesome)

Preparatory work

Install the latest nodejs using PPA:

curl -fsSL https://deb.nodesource.com/setup_20.x | bash - apt-get install -y nodejs 

---

Note 1: When I wrote this article, the latest nodejs installed by apt was 18.13. If you use apt to install the 18.13 version directly, you will get an error when installing umami later, so I changed to the above PPA installation method:

Note 2: This method has already installed npm, so there is no need to use apt to install it separately. Of course, if you have to use apt to install it separately, it is not impossible, but you will encounter a lot of errors. Don't ask me how I know.

---

Install git

apt install git

Install Yarn

npm install -g yarn

Install gm2

npm install -g pm2

Install and initialize the database

Use mariadb (use with caution, pending confirmation)

The instructions on github support Postgresql (minimum supported version: v12.14) and MySQL (minimum supported version: v8.0). When I used docker-compose to install before, I used Postgresql, but I was a little unhappy because I use mariadb more often and there is a ready-made one. Since MySQL is supported, I will use mariadb as the database first this time (which turned out to be a failure).

---

If there is no existing mariadb database, it is still recommended to deploy mariadb using docker (in fact, from the perspective of obsessive-compulsive disorder, the mariadb database should also be installed using source code, but the database deployed by docker is indeed too convenient in terms of operation and maintenance, so this time even if I am seriously ill with obsessive-compulsive disorder, I can only bear it):

 docker run --name=mariadb -d --restart=always \ -p 3306:3306 \ # Map the host to port 3306 inside the container, set according to your actual environment -v /docker/mariadb/db:/var/lib/mysql \ # Map the directory on the host that needs to be mounted to the directory inside the mariadb container -e MARIADB_ROOT_PASSWORD=yourpassword \# mariadb database root password mariadb:10.11

Note 1: Remove # and the following comments by yourself when using
Note 2: If you are more familiar with Postgresql, you can use it. It doesn’t matter which one you use (it turns out that it does matter).

---

For the steps to initialize the database, see the article:Tips and tricks: Create a new empty database and grant permissions to corresponding usersHere we assume that we create a new library named "umami" and create an account with the username and password of umami and full permissions to the umami library.

Using Postgresql

The docker run format to deploy postgresql commands is as follows:

docker run --name postgres -d --restart=always \ -p 5432:5432 \ -e POSTGRES_USER=umami \ -e POSTGRES_PASSWORD=umami \ -e POSTGRES_DB=umami \ -e PGDATA=/var/lib/postgresql/data /pgdata \ -v /docker/postgres/data:/var/lib/postgresql/data \ postgres:15.4

Note: The most convenient thing about using postgresql is that the initialization process can be completed using the -e parameter, but I think mariadb should also be able to do it. I didn’t pay much attention to it before, because many libraries were initialized manually by myself later, and I’m used to it. I’ll study it again when I have time.

Install umami

Choose an installation directory. I usually install it in the root folder:

cd ~ git clone https://github.com/umami-software/umami.git cd umami yarn install

success:

Create a new .env file and fill in the database information in the following format:

DATABASE_URL=mysql://umami:umami@localhost:3306/umami #Select mariadb or mysql DATABASE_URL=postgresql://umami:umami@localhost:5432/umami #Select postgresql TRACKER_SCRIPT_NAME=tangwudi #This is very important. You can customize the js script name without adding the js ending. I will explain it later.

Note 1: localhost:3306 means that the database and umami program are on the same device. If the database here is on a different host, localhost needs to be changed to the IP address of the host.
Note 2: DATABASE_URL selects different database links according to the database you choose. You can choose one of the two.

Build umami:

yarn build

---

Attention!!!! If you use mysql or mariadb, I encountered the following error:

Solution: Open the package.json file in the umami folder (the package.json file installed in source code mode is equivalent to the docker-compose.yml file in the docker-compose installation mode. If you need to change the default configuration information such as the port, modify it here):

Change the content of line 23 in the red box in the above picture:

"build-db": "npm-run-all copy-db-files build-db-client",

Replace it with the following:

"build-db": "npm-run-all copy-db-files build-db-client resolve-db", "resolve-db": "prisma migrate resolve --applied '05_add_visit_id'",

Then re-run the commandyarn buildJust build it.

---

Build Success:

Update the browser list:

npx update-browserslist-db@latest

Test whether umami can start normally:

yarn start

For formal use, you need to enable umami in the background:

pm2 start yarn --name "umami" -- start

Automatically start pm2 at boot time and save the currently running umami process:

pm2 startup pm2 save

---

Note: If you want to stop and delete related processes using the pm2 process manager, use the following command:

pm2 list # to see which pm2 process ids are there pm2 stop process id pm2 delete process id

In fact, it just uses pm2 to replace the regular service settings, which feels quite useful.

---

Then use directlyhttp://deployment machine IP:3000There are two results depending on the selected database, at least what I encountered:
1. mysql or mariadb
After adding the site, the dashboard directly displays an error:

What is the problem? Is it that the problem is being solved before?yarn buildWhen the P3009 error was reported, I actually referred to the method of dealing with similar problems when using Vercel to build umami. Is this method not suitable for local conditions? Because the actual processing method affects the generation of the database, and the same problem occurs whether I change to mysql or mariadb. In order to verify my judgment, I changed to postgresql. Unexpectedly, there was no error during the build, and it passed directly. Moreover, there was no "error" error in the dashboard later.

I looked at the deployment using source code online, and found that many problems such as P3005 and P3009 occurred during the upgrade (this is because the commands of mysql 5.7 are incompatible with the new 8.x, and the corresponding commands in the sql file used during initialization need to be modified). Now I think that the problem with umami deployed using docker-compse was also because I tried to upgrade it not long ago. It seems that you should be cautious when upgrading umami, especially when using msql/mariadb, because it involves some command changes from mysql 5.7 to the new version 8.x, so it is particularly easy to have problems.

2. postgresql
This is fine:

Use docker to build umami (recommended for general use, I will use it next time)

Create docker-compose.yml file

Create a docker-compose.yml file in the corresponding directory, I am in /docker/umami:

mkdir -p /docker/umami/data #data directory is used when postgresql is mounted later, so it is created in advance cd /docker/umami vim docker-compose.yml

---

When I used the docker-compse method to build umami, the image address used was relatively old and it seems that it is no longer accessible (image address:docker.umami.is/umami-software/umami:postgresql-latest), this time I use the latest official one. According to the yml provided by umami on github and the yml in my previous article on using docker-compose to build umami, I made some modifications to the content. The following is the latest docker-compose.yml file content:

--- version: '3' services: umami: image: ghcr.io/umami-software/umami:postgresql-latest ports: - "3000:3000" environment: DATABASE_URL: postgresql://umami:umami@localhost:5432 /umami DATABASE_TYPE: postgresql APP_SECRET: replace-me-with-a-random-string TRACKER_SCRIPT_NAME: tangwudi depends_on: db: condition: service_healthy restart: always healthcheck: test: ["CMD-SHELL", "curl http://localhost:3000/api/heartbeat"] interval: 5s timeout: 5s retries: 5 db: image: postgres:15-alpine environment: POSTGRES_DB: umami POSTGRES_USER: umami POSTGRES_PASSWORD: umami volumes: - ./data:/var/lib/postgresql/data restart: always healthcheck: test: ["CMD-SHELL", "pg_isready -U {POSTGRES_USER} -d{POSTGRES_DB}"] interval: 5s timeout: 5s retries: 5

---

Note: Compared with the yml content in the previous article, it can be seen that a sql file (schema.postgresql.sql) required to initialize postgresql has been cancelled because it is no longer needed. The initialization content has been built into the umami program. In the official folder of github, you can actually see the initialization steps and content of the postgresql database:

mysql/mariadb is similar:

---

Pull the image

docker pull ghcr.io/umami-software/umami:postgresql-latest docker pull postgres:15-alpine

Pull up the container

cd /docker/umami docker-compose up -d

Then, just like the previous steps of deploying using source code, usehttp://deployment machine IP:3000way to access.
After testing, it is indeed the latest version:

---

Note: I spent 2 days researching the source code deployment, mainly because I chose mariadb as the database at the beginning. However, even if you become proficient and don’t take detours, it still takes about ten minutes to deploy once, not to mention the strange problems that may be caused by differences in different environments. If you use docker-compose to build, is it 30 seconds? ? So for umami, it is recommended that ordinary people do not bother with the source code when building.

---

Optimize umami configuration

For detailed initialization configuration, please refer to my previous article. I will not repeat it here. I will only talk about a few key points of optimization that have not been mentioned before.
1. Prevent being blocked by anti-ad plugins

The default js script name of umami is "script.js" or "umami.js". This name is very clear and not very hidden. In order to prevent being directly blocked by anti-ad plug-ins, you need to disguise it and change it to a less easily discovered vest. For example, I changed it to "tangwudi"~. If you install it through docker-compose, you need to add environment variables in "docker-compose.yml":

environment: TRACKER_SCRIPT_NAME: tangwudi

If you install it using source code, you need to add it in the .env file

TRACKER_SCRIPT_NAME=tangwudi

Note: The source code deployment method must be runningyarn buildIt is only useful if you add it before, otherwise, if you add it later, the tracking code will become tangwudi, but you can't find the webpage error when you access it. It is only useful to access "script.js", forcing me to rebuild it again; similarly, when installing using docker-compose, you need to rundocker-compose up -dBefore the command.
2. Put the js script (tangwudi in this example) on the object storage

In essence, umami's statistical method is to run a js script through the browser, and then transmit the results back to the host where umami is deployed to complete the statistical work. Therefore, in theory, it doesn't matter where the script is placed, as long as the browser can read it. For example, I can put the js script on cloudflare R2 (if it is a domestic server, it can be placed on the COS of a domestic cloud provider, such as Tencent's COS), and enable CDN, which can minimize the back-to-source requests. This is achieved by modifying the "src" part of the tracking code:

src="https://r2.tangwudi.com/tangwudi" # Assume r2.tangwudi.com is the custom domain name of my r2 bucket

However, I think that if there is a CDN, this can be cached directly on the CDN. After all, it is just a static content and will not be frequently returned to the source. Moreover, this option must be used in conjunction with the next optimization parameter "data-host-url", which feels a lot more troublesome, because if there is no "data-host-url" parameter, the address in src will be directly used as the address of the umami server for receiving statistical data, which saves a lot of trouble.

3. Specify the server address that actually receives the statistical data

data-host-url="https://umamiserver.tangwudi.com" # actually deploys the server access domain name of umami

This is generally used in conjunction with the previous parameter, depending on personal choice.

4. Only display the collection information of a certain domain name

data-domains="blog.tangwudi.com"

This option is actually very useful in theory. For example, my wordpress can be accessed directly with the intranet address at home, or with the tailscale address outside. Before I added this option, the access to these addresses would be displayed on the monitoring page, which also triggered an SSRF (server-side request forgery) attack on me because the intranet address 192.168.xx of my wordpress server was displayed on the monitoring interface.

---

Note: Why do I say theoretically? Because I feel like I was fooled. The sharing link generated in the end is like this:

It is like adding "/blog.tangwudi.com" after the original sharing link???, the key is that you can directly remove it and still access it. After removing it, can you see all the addresses as before? ?

---

Finally, my tracking code is as follows (only the first and fourth optimizations are used):

Upgrading umami

Source code deployment method

Three steps:

git pull #Operate yarn install yarn build in the umami folder

docker-compose way

docker compose pull docker compose up --force-recreate

It seems very simple, but according to the English posts I read in recent days to solve the strange problems caused by using the mariadb library and the trigger for me to write this article, you should be cautious when upgrading umami, especially when the version span is large (a smaller version may not be safe). It is recommended to back up the database content first.

Summarize

At least for umami, the source code deployment method is indeed too troublesome (in order to verify whether the error problem of the dashboard when using the mariadb database was caused by environmental reasons, I later built a brand new environment of debian12 lxc, so now umami is actually running on lxc, and even postgresql deployed by docker, I am too lazy to bother with it anymore), and docker-compose deployment is recommended.

The new version is better than before, with an additional display of visit count:

There is one more behavior category, I don’t know what it is, let’s wait a day and see:

You can use the following link to see the actual effect of the new version of umami:New version of umami demo link.