1 When embedding moves from experiment to foundational capability

In previous articles introducing RAG, I mentioned that the core tool for vectorizing segmented text is the ”embedding model“ (see article:Understanding RAG from Scratch (Part 1): Principles and Complete Process Analysis), and also introduced in a subsequent hands-on article: the steps to implement a simple local knowledge base using Chatbox and a self-built embedding model via Ollama (see article:Building a Self-Hosted Embedding Model with Ollama + Chatbox Knowledge Base in Practice)，

At that stage, embedding was more of an “experimental capability”: it was used to verify whether the workflow was feasible, whether the model's performance was acceptable, and whether local deployment could functionally replace cloud services. As long as a complete workflow could be run successfully once, issues like stability, runtime environment, and long-term maintenance were often not prioritized highly.

However, as use cases gradually increase, the role of embedding is also quietly changing: whether it is a continuously maintained local knowledge base, or subsequently introduced RAG, full-text search, similarity calculation, or even more engineering-oriented knowledge graphs or AI-enhanced blog features, embedding is no longer a temporary step that “only needs to be generated once,” but is gradually evolving into aresident foundational capability。

When embedding shifts from “occasional use” to “repeated invocation,” some previously ignored issues begin to surface: for example, should it continue to rely on a desktop system (such as using an Ollama instance on macOS as a long-term runtime environment)? Should it share the same runtime boundary with the daily-use operating system, toolchains, and desktop environment? If system updates, environmental changes, or tool upgrades introduce uncertainty, will this foundational capability be accidentally disrupted?

From engineering perspective, embedding has very distinct characteristics: it is a non-autoregressive computing process, not on the user's real-time interaction path, and hashigher requirements for stability, reproducibility, and consistency of long-term running behavior. In other words, it is closer to “infrastructure” rather than an “application” suitable for long-term running on a personal desktop system.

It is precisely based on this judgment that I began to re-examine the deployment method of embedding: rather than letting it continue to rely on desktop environments like macOS, drifting passively with system upgrades and tool changes, it is better to explicitly decouple it and place it in an independent environment with clear runtime boundaries, predictable behavior, and a controllable update pace, existing as a dedicated service for the long term.

Meanwhile, from the perspective of resource characteristics, the embedding function does not heavily rely on GPUs, and its overall demand for computing resources is relatively controllable; in most practical scenarios, even a standard Intel CPU or similar configuration is sufficient to support the vectorization needs of daily knowledge base articles. This makes deploying embedding independently on a general-purpose computing node a well-balanced choice in terms of cost and stability from an engineering standpoint.

Therefore, I ultimately deployed a fixed-version Linux system via LXC on PVE on my home Intel CPU mini PC, specifically to host Ollama's embedding model service. This reduces the impact of system updates and runtime environment changes on the knowledge base construction workflow, while ensuring the controllability and reproducibility of the service operation.

In the subsequent chapters, starting from the analysis of engineering characteristics, I will explain why embedding is suitable for independent deployment, and combine it with the actual environment to fully demonstrate how to deploy a Linux environment dedicated to providing embedding services via LXC in PVE, laying a stable foundation for subsequent knowledge bases and RAG applications.

**Item 9:**

From a pure engineering efficiency perspective,the most economical and stable way to implement the embedding function is actually to directly use embedding models provided by mature commercial services, such as the embedding API provided by OpenAI. These services are far superior to self-built solutions in terms of model quality, stability, and long-term maintenance costs, and they almost completely avoid the runtime uncertainty issues discussed in this article.

However, in practical use within China, this path is not always feasible. On one hand, there are objective restrictions on bank cards and settlement methods in the payment process; on the other hand, even if non-mainstream access methods exist, the availability period and stability of official APIs are difficult to guarantee. For a foundational capability like embedding, introducing additional uncertainty and maintenance costs is actually not cost-effective. Moreover, some commonly used third-party API providers (such as OpenRouter, OhMyGPT, etc.) almost exclusively offer large language model interfaces, while embedding models are typically not provided due to limitations in call volume, computing power, and application scenarios.

Because of this, I chose to give up relying on external commercial services and instead deploy a Linux environment via LXC in PVE to build my own Ollama embedding model service. This choice is not about pursuing optimal performance or results, but is based on a comprehensive consideration ofcontrollability, sustainability, and overall cost balance.

2 Implementing embedding services in a controlled environment

1 Creating an LXC in PVE “solely for the embedding service”

After clarifying the need to decouple the embedding capability from the desktop system, the next question is not “how to deploy Ollama,” but ratherwhat kind of environment this embedding capability should run in。

In my practical scenario, this question can be broken down into three more specific engineering judgments:
Why use LXC?
Why choose Debian 12?
How much resource does this environment actually need to be considered “reasonable”?

Why use LXC instead of VM or container orchestration solutions

The operational characteristics of the embedding service are actually very clear: resident operation; long-term stability; no pursuit of elastic scaling; no need for frequent reconstruction or destruction. It is more like aninfrastructure capability, rather than a one-time task or a short-lifecycle service.

Under this premise, compared to full virtual machines, the advantages of LXC are very direct:

• It provides afull Linux system perspective, rather than a highly abstracted runtime
• It can natively use systemd to manage Ollama as a long-running system service
• Lower resource overhead, with startup and maintenance costs closer to a “real server”

As for Docker or more complex container orchestration solutions, they seem over-engineered in this embedding scenario: there is no need for multi-instance scaling, rapid image iteration, or complex dependency isolation, and introducing an extra abstraction layer only increases the cognitive burden of long-term maintenance.

Therefore, LXC is not a “technically more advanced” choice here, but rathera choice that is closer to the actual running state of the embedding service。

Why Choose Debian 12

After deciding to use LXC, the choice of operating system actually follows the same set of judgment logic:Stability over new features。

There are several main reasons why Debian 12 became my final choice:

• Stable release cadence, clear lifecycle, and predictable long-term behavior
• Extremely high popularity in server and container environments
• Ollama and its underlying dependent components (such as llama.cpp) have higher compatibility and test coverage on Debian-based distributions

For the embedding service, the operating system is not the object of study, but merely aplatform that carries capabilities. Choosing Debian 12 is essentially choosing a path of “slowest change and fewest surprises.”

Note: Choosing Debian is just my personal habit because I am most familiar with it. If you are familiar with other Linux distributions, choosing the one you are familiar with is just as good, no need to overthink it.

Why Resource Specifications Do Not Need to Be Large

The final question is resource allocation. The computational characteristics of embedding have already been analyzed in Chapter 2: it is a non-autoregressive, linear computational process, not on the main path of real-time user interaction, and there is no continuous high-concurrency pressure.

Therefore, the resource allocation goal for this LXC is not to “run as fast as possible,” but rather:Stable, predictable, and capable of running long-term without frequent intervention。

In actual deployment, I allocated the following specifications to this LXC as a reference configuration:CPU: 2 Cores；Memory: 4 GB；Disk: 20 GB。

This configuration is completely sufficient for the embedding service: the overall time consumption of embedding is mainly affected by the input text length and the computational characteristics of the model itself, rather than system resource bottlenecks. In daily use, even processing multiple medium-length Chinese articles consecutively will not put continuous pressure on the CPU or memory.

In other words, configuring excessively high-spec resources for the embedding service will not only bring no substantial benefits, but may also blur its true positioning in the entire system—it should be a“basic service that ”works quietly", rather than a performance node that needs significant attention and tuning.

It should be noted that such resource allocation is not “extreme compression,” but a conscious choice based on the actual use cases of embedding. In PVE's LXC environment, the cost of resource adjustment and expansion is inherently very low. If the model scale or usage patterns change in the future, it can be adjusted then.

After clarifying the above three prerequisites, the creation process of the LXC itself becomes relatively simple:

I have explained the specific steps on how to create an LXC in PVE in many previous articles, so I won't repeat them here.

2 Deploying Ollama in a Debian 12 LXC

In the previous section, the role of this LXC has been very clearly defined: it is not a general computing node, nor is it an interaction entry point, but rather along-term resident component that exists solely for the embedding service. Under this premise, the responsibility of Ollama here also becomes extremely single-minded—only responsible for loading the embedding model and providing a stable vectorization interface to the outside.

Precisely because the goal is clear enough, the deployment of Ollama does not require complex installation methods. From an engineering perspective, the most important thing here is not “playability” or “customizability,” but the predictability of behavior: whether the installation process is clear, whether the running method is stable, and whether the upgrade path is clear.

The official installation script provided by Ollama fits this orientation perfectly. It installs a pre-compiled official binary, runs as a systemd service by default, does not depend on GPU or platform features, and does not make deep invasive modifications to the system. For a basic service running in an LXC with a single responsibility that needs to run stably for a long time, this “restrained” installation method is actually the most appropriate choice.

Therefore, in this LXC, you can complete the entire installation process directly using the official script (recently Ollama made an adjustment to the installation package format, changing from tar.gz to tar.zst, but since many Linux distributions do not come with zstd by default, you need to install it yourself):

apt update
apt install -y zstd
apt install -y curl
curl -fsSL https://ollama.com/install.sh | sh

apt install -y zstdhttp://192.168.1.1:8080as an example:

export http_proxy="http://192.168.1.1:8080"
export https_proxy="http://192.168.1.1:8080"
curl -fsSL https://ollama.com/install.sh | sh

curl -fsSL https://ollama.com/install.sh | sh

In a normal network environment, you can complete it directly using the installation script (as shown in the figure above). Unfortunately, the domestic network environment happens to be an abnormal network environment, so you need to use "science" or "magic" (VPN/proxy), with the proxy address set as

export http_proxy="http://192.168.1.1:8080"

export https_proxy="http://192.168.1.1:8080"

curl -fsSL https://ollama.com/install.sh | sh It is worth noting that if you directly use a global transparent proxy on the egress router, there may also be issues, because curl | sh has very strict network requirements: the download must be complete, TLS cannot be interrupted, and the shell cannot receive a half-finished script. The router's transparent proxy may experience instability such as resets, fragmentation, or IPv6 fallback at the TCP/TLS layer, causing the script to fail midway.Therefore, the safest approach is to explicitly specify the proxy through environment variables, so that all download requests within the script go through the same stable channel.After the installation is complete, Ollama will automatically start as a system service. At this point, you can check its running status via the systemctl command to confirm that the service is working normally:Note 1: If you need to update the version of Ollama, you only need to re-run the installation script. Note 2: When deploying Ollama inside the Debian 12 LXC container in this article, thedefault CPU-only running mode

3 (Optional) Configuring Ollama's listening address and model download directory

After completing the Ollama installation in Debian 12's LXC, by default, its service only listens on 127.0.0.1:11434. This address means that only the inside of the LXC can access it. If you want to access the service from the host machine or other devices on the local network, you need to change the listening address to the network card IP allocated to the LXC (such as 192.168.x.y).

Ollama's listening behavior is not exposed through a configuration file, but is controlled through the environment variables of the systemd service. Its service file path is:

/etc/systemd/system/ollama.service

It is empty by default, and you need to add the following content yourself:

Environment="OLLAMA_HOST=192.168.x.y:11434"  # 监听LXC网卡IP，端口可自定义
Environment="OLLAMA_ORIGINS=*"            # 允许所有跨域请求

As shown in the figure below:

mkdir -p /ollama/models
chown -R ollama:ollama /ollama
chmod -R 755 /ollama

mkdir -p /ollama/models

Environment="OLLAMA_MODELS=/ollama/models"

As shown in the figure below:

systemctl daemon-reload
systemctl restart ollama

Environment="OLLAMA_MODELS=/ollama/models"

ss -ntpl
netstat -ntpl

systemctl daemon-reload

systemctl restart ollama

4 Downloading and running the embedding model

4.1 Downloading the embedding model

You can verify whether it is running normally by checking Ollama's listening status via ss -ntpl (or netstat -ntpl):

ss -ntplnomic-embed-text netstat -ntplnomic-embed-text When Ollama is running normally, the output is as follows:

ollama pull nomic-embed-text

At this point, Ollama has been stably running as a system-level service in Debian 12's LXC. At this moment, it is still just a "runtime environment" and has not loaded any models yet. The next section will truly introduce the model files required for embedding and let this LXC start undertaking actual vectorization tasks.

In the previous steps, Ollama has been stably running as a systemd service in Debian 12's LXC and listening on the specified network address. However, it should be noted that at this time, this environment is still just an "empty runtime"—it has not loaded any models yet, nor has it truly undertaken vectorization tasks.

In the list of models currently provided by Ollama, there are actually not many choices available for embedding. In contrast,

is considered a relatively mature and widely used embedding model, and its performance in Chinese scenarios is also relatively stable. Therefore, when migrating to the Linux environment this time, I did not change the model, but continued to use

export OLLAMA_HOST=192.168.10.104:11434

However, before actually executing this command, there is a very easily overlooked but crucial prerequisite that needs to be explained: Ollama is designed as a typical client-server architecture—ollama serve is responsible for the long-running service process, while commands like ollama pull and ollama embed themselves are just client calls. They do not automatically start the service, but communicate with the already running Ollama service through network requests.

OLLAMA_HOST=192.168.10.104:11434 ollama pull nomic-embed-text

This means that if you execute ollama pull directly in the terminal, the client will still try to connect to the default 127.0.0.1:11434, resulting in the "could not connect to ollama server" error message, even though Ollama is actually running normally on the correct address:

4.2 Loading and running the embedding model

Therefore, before pulling the model, you need to explicitly specify the Ollama service address in the current shell first, for example:export OLLAMA_HOST=192.168.10.104:11434Or temporarily specify it in a single command:OLLAMA_HOST=192.168.10.104:11434 ollama pull nomic-embed-text。

After clarifying that the client and server are using the same address, the model download can proceed normally:The responsibility of the ollama pull phase is limited todownloading and storing the model files locally

, and it will not immediately load the model, nor will it actively spin up any new running instances. Whether it is model downloading or subsequent calls, the ollama CLI is essentially just a client, and it always relies onthe already reachable Ollama ServerThe actual model loading occurs

OLLAMA_HOST=192.168.10.104:11434 ollama run nomic-embed-text "Hello world"

. Once an embedding model is called for the first time, Ollama will load it into memory in the background and keep it resident; as long as the instance is not recycled, subsequent embedding requests will reuse the same running instance without needing to repeatedly initialize the model for each request.

Therefore, verifying whether the model is "truly available" is not about checking whether the model pull was successful, but about checking

whether a complete call chain can be completed. Inside the LXC, you can verify this with the following minimal command:OLLAMA_HOST=192.168.10.104:11434 ollama run nomic-embed-text "Hello world"

Note: The current ollama CLI does not provide a separate embed subcommand, so ollama run is used here as the minimal verification method

3 Summary and afterword: The runtime location of embedding as a foundational capability

If the command can normally return a set of vector data, it means three things are true at the same time: the Ollama Server is in a connectable state; the embedding model has been successfully loaded into memory; and the entire embedding path from client call to vector output has been established. The normal input is shown in the figure below:

In this process, I deliberatelydid not run any chat models, Web UIs, or RAG logic in this LXC. Its responsibility is strictly limited to one thing: providing embedding capabilities stably and predictably. Such a trade-off is not a compromise on functionality, but an engineering layered choice—embedding is more like a piece of infrastructure rather than an experimental object that requires frequent intervention and debugging.

At this point, this Debian 12 LXC truly begins to undertake vectorization tasks.Reducing long-term risks. This adjustment does not aim for optimal performance or results, but prioritizes ensuring predictable behavior and sustainable operation.

It should be noted that, where conditions permit, directly using embedding APIs provided by mature commercial services remains a choice with higher engineering efficiency and stability. The solution discussed in this article is mainly applicable to scenarios where relying on external services is impossible or inconvenient, while also hoping that the embedding can run stably over the long term.

On this basis, model effectiveness, vector database selection, and the specific implementation of RAG only then make sense to discuss further. At least in my practice, resolving the issue of where the embedding runs first is far more important than prematurely stacking upper-level structures.