1 How does AI compare vectors?

In the previous article (see:AI Semantic Space Exploration (II): From a Word to an Article—How Semantic Vectors Are Generated), we already know that whether it is a word, a sentence, or an entire article, it can be converted into a semantic vector through an Embedding model and enter the same semantic space.

But vector generation is not the end. Once different contents are converted into a unified form of vectors, a new question arises:How should AI determine whether these contents are semantically close?

For humans, we can judge whether two pieces of content are similar based on experience and intuition; but for AI, it needs to analyze the characteristic differences between vectors through mathematical methods to determine their closeness in the semantic space.

This problem seems to be just a vector comparison issue, but in fact, it exists in almost all AI applications that rely on semantic understanding. For example, when clicking on “Related Articles” recommendations on my blog's article interface, why can the system recommend related topics based on the article content instead of simply recommending articles under the same category? Why can semantic search find content related to the meaning of the question? And why can RAG (Retrieval-Augmented Generation) find the most suitable reference text from a large amount of data?

Although these capabilities have different application scenarios, they all rely on the same basic process behind them: converting content into vectors and then comparing the differences between these vectors to determine whether they have similar semantic expressions. So, what method should be used to compare the differences between vectors?

2 When we compare vectors, what are we actually comparing?

To compare vectors, we first need to think through a fundamental question:When we say “comparing two vectors”, what are we actually comparing? To understand this, we might as well re-examine “vectors” themselves.

In the previous article, we already know that Embedding converts a piece of text into a semantic vector, but in many people's minds, a vector is still just a point—as if it were merely a coordinate in the semantic space. However, mathematically speaking, a vector is usually represented asan arrow starting from the coordinate origin. It not only records the position where the arrowhead is located, but also contains information such as length and direction:

As shown in the figure above: position represents the end coordinate of this arrow in space, which is where it lands; length represents how long this arrow is, which is its own scale; direction represents where this arrow points.

Since a vector itself contains so much information, the so-called “comparing two vectors” naturally cannot have only one method.

For example, if what we care about isposition, the most direct method is to calculate the distance between them. However, the distance here is not the distance between the two arrows as a whole, but ratherthe distance between the two arrowheads (which are the vector endpoints) in space. The closer the arrowheads, the closer the positions of the two vectors in space; the farther the arrowheads, the farther apart they are. This is also the way of comparison we are most familiar with in the real world.

If what we care about islength, then we are comparing whether the scales of the two vectors themselves are similar. Although some vectors point in the same direction, their lengths may differ greatly; other vectors may have very close lengths but point in completely different directions.

And if what we care about isdirection, then the focus changes again. At this point, we care more about whether the two arrows point in similar directions, rather than just their lengths or the distance between the arrowheads. Therefore,different comparison methods are essentially choosing different information in the vector as the basis for judgment.

Returning to the question raised in Chapter 1, what we really want to solve is not “whether two vectors are close in geometric space”, but “whether two contents are semantically similar”. Since semantics have already been mapped to the vector space, the key lies in: among these observable geometric features, which relationship best reflects the degree of semantic closeness?

3 Why does AI often focus on vector direction when determining semantic similarity?

In the previous chapter, we already know that a vector does not just represent a position point in the semantic space. It simultaneously contains multiple pieces of observable information: the position of the arrowhead, the length of the vector itself, and the direction it points to. There is no absolute hierarchy among these pieces of information; different information actually corresponds to different questions.

So, when our goal is to determine whether two contents are semantically similar, how should we understand the meanings represented by each of these pieces of information? To answer this question, we first need to observe what information position, length, and direction can respectively provide in vector comparison.

First, let's look at the positional relationship.

If our goal is to analyze the overall distribution of a large amount of content in the semantic space, then the position of the vector is of great value. For example, on my blog, articles introducing Cloudflare Tunnel, Cloudflare Access, and Cloudflare Zero Trust, because they all revolve around Cloudflare-related technologies, their corresponding vectors may form a relatively concentrated area in the semantic space. On the other hand, topics like home network architecture and database optimization may be distributed in different positions.

In this case, AI focuses on “which contents are clustered together in the semantic space”, so the positional relationship can help complete tasks such as topic classification and automatic clustering.

However, when the task becomes determining whether two contents are semantically similar, we cannot draw a conclusion based solely on “closer raw distance”. This is because whether the distance in the vector space can accurately reflect semantic relationships also depends on what objectives the Embedding model adopted during training, and what information this task hopes to retain.

Therefore, at least in the semantic similarity scenarios discussed in this article, the raw distance between vectors is usually not suitable as the primary basis for judgment.

Next, let's look at length.

Vector length sometimes also carries certain additional features calculated by the model. For ease of understanding, we can temporarily think of it as the “overall scale” of the content: even if two pieces of content discuss the same topic, they may have different lengths due to differences in expression, information scope, etc.

For example, of two articles about Cloudflare Tunnel, one only briefly introduces what it is and what problems it solves; the other not only introduces basic concepts but also further explains the working principles, network flows, configuration methods, and actual deployment experience. Due to differences in expression scope and information scale, there may also be certain differences in the length representations of their corresponding vectors.

However, for semantic similarity judgment, the amount of content is not the most core issue. An article being written in more detail does not mean it discusses a completely different direction from another article. Therefore, although length may reflect certain expression characteristics, it cannot directly represent the degree of semantic closeness.

So, the remaining question is: if we want to determine whether the core meanings expressed by two contents are close, which kind of information in the vector is more in line with this goal? The final answer is direction. Compared to position and length, direction is closer to the issue that semantic similarity judgment focuses on. Because in the semantic space formed by Embedding, the vector direction reflects the semantic expression trend presented by the content after being converted by the model.

For example, a brief article introducing home servers and a detailed article discussing home data center architecture, although containing different scales of information and different scopes of expression, may still keep their directions close in the semantic space if they revolve around similar issues.

However, a new question arises here: although direction is the information we want to compare, length still exists in every vector. So, when two vectors have different lengths, will this difference affect direction comparison? If so, how should AI eliminate this influence?

4 Why is normalization needed? How does normalization unify length?

In the previous chapter, we already learned that for semantic similarity judgment, we prefer to compare the directional relationship between vectors. But the problem is that direction does not exist in isolation; every vector also contains length information. Therefore, before comparing directions, we first need to solve a problem: how to eliminate the impact of length differences?

To understand this problem, we can start with a more intuitive proportional relationship. Suppose we compare the body structures of two people:

Person A is 180 cm tall and weighs 72 kg
Person B is 150 cm tall and weighs 60 kg

If we only look at absolute values, it is easy to think that A is larger in every dimension. But if we focus on “whether the structure is similar,” then what really matters is no longer the total amount, but the proportional relationship between different dimensions.

Calculations show that:

A: 72 / 180 = 0.4
B: 60 / 150 = 0.4

The ratios of the two are exactly the same. This shows that although their overall scale is different, the internal structural relationship has not changed; they are simply in different scaling states.

When this logic is transferred to vector space, it is similar. For example, two two-dimensional vectors:

A = (2, 1)
B = (4, 2)

It can be seen that B is just a scaled-up version of A. The ratio of the components of the two vectors remains consistent:

A: 2 / 1 = 2
B: 4 / 2 = 2

Therefore, although they have different lengths, the directional relationship they express has not changed.

The problem thus becomes clear: if different vectors are just in different overall scaling states, then when comparing them, we first need to eliminate this scale difference; otherwise, the length factor may affect the comparison results.

This is precisely the problem that Vector Normalization solves. The core idea of normalization is very simple: for any vector, first calculate its own length, and then use this length as a scaling baseline to scale the entire vector proportionally, so that all vectors ultimately have the same length (usually unified to 1):

Formally, it can be expressed as:

v’ = v / |v|

For a two-dimensional vector:

(x’, y’) = (x / √(x² + y²), y / √(x² + y²))

It is important to note here that normalization does not adjust each dimension individually, but scales the entire vector proportionally. Therefore, although the specific values of each dimension change, the proportional relationship between them remains unchanged.

In other words, normalization converts a vector representation that originally contained both “direction” and “length” into a form more suitable for directional comparison. Once all vectors are at the same length standard, we can further judge the semantic similarity relationship between them by measuring how close their directions are.

5 How to measure whether the directions of two vectors are close?

After normalization, the length differences between vectors have been eliminated. Therefore, only one problem remains to be solved: how to measure whether the directions of two vectors are close?

If we return to the previous arrow metaphor, the lengths of all arrows are now the same. Therefore, when comparing two vectors, what we really need to observe is no longer which one is longer, but the angle relationship formed between them. The smaller the angle, the closer the directions of the two vectors; the larger the angle, the more obvious the difference in the directions they express:

Mathematically, a classic method to measure how close the directions of two vectors are is to calculate the cosine of the angle between them. This is also why Cosine Similarity came into being.

Cosine similarity does not directly calculate how far apart two vectors are, but converts “directional closeness” into a comparable numerical value through the cosine of the angle. When the directions of two vectors are exactly the same, the angle between them is 0°, and the cosine value is 1, indicating high directional consistency; when the directions of two vectors are completely opposite, the angle is 180°, and the cosine value is -1; and when two vectors are perpendicular to each other, the angle is 90°, and the cosine value is 0:

Therefore, cosine similarity actually completes a transformation: it converts “directional closeness,” which was originally difficult to measure directly, into a clear mathematical indicator.

That is to say, after semantics are mapped to vector space, the question of “whether two contents are similar” has essentially been redefined as a geometric problem: whether the directions pointed to by two vectors in space are close. And cosine similarity is the mathematical tool used to answer this question.

Here, some readers might have a new question: Since we already know that we are comparing the angle between two vectors, why do we use the “cosine value” in actual calculations instead of directly calculating the angle itself?

The reason is that, for computers, directly processing angles is not the most convenient way, whereas the cosine value can reflect the directional relationship between two vectors while being computationally simple and numerically stable. Therefore, in vector retrieval, semantic search, and most embedding applications, cosine similarity has become a commonly used method for measuring vector similarity.

It is worth noting that cosine similarity itself is not an AI-exclusive algorithm; it is merely a mathematical tool in linear algebra used to describe the directional relationship between vectors. The key to its application in semantic understanding is that embeddings convert text into vectors, and normalization eliminates differences in length, allowing this calculation method—originally belonging to geometric space—to be naturally used to measure semantic closeness.

6 Afterword: Vector similarity is just a projection of semantic understanding

In the previous sections, we deduced all the way from “how semantics are represented” to “how vectors are compared,” finally landing on the seemingly simple calculation method of cosine similarity. At this point, a complete chain has been established: text is converted into vectors, vectors have positions, lengths, and directions in space, and semantic similarity is transformed into a judgment of how close their directions are.

But if we stop here, it is easy to create a misunderstanding: that semantic understanding itself is merely geometric operations in vector space. In fact, vector space is just a way for us to observe and process semantics; it is more like a “projection” of complex semantic relationships.

This “projection” can be understood this way: originally complex semantic phenomena are compressed into a mathematical space, and then their relationships are approximately described through geometric concepts such as position, distance, angle, and direction. This process is highly effective in engineering, but it is not equivalent to semantics itself; rather, it is a computable form of expression.

Therefore, “similarity” between vectors is essentially just a mathematical mapping of semantic relationships in vector space. The reason cosine similarity has become the most common method in semantic comparison is that it corresponds to a core assumption established earlier: in many semantic similarity tasks, we care more about whether the expressive directions of two contents are close, rather than their absolute positions or differences in scale within the space.

However, it is worth noting that cosine similarity is not the only reasonable way of comparison. Different tasks focus on different information: some scenarios care more about the distance between vectors, while others aim to preserve the information carried by length. Therefore, which specific comparison method to adopt still depends on the model and the task objectives.

Returning to the original question: How does AI judge whether two vectors are similar? Now we can put it more accurately: AI does not directly understand the concept of “similarity”; instead, within a designed semantic space, it chooses an appropriate mathematical method to approximate the relationship between contents.

Therefore, vector space is not semantics itself, but a computable expression of semantics in the mathematical world. It maps originally complex and hard-to-describe semantic relationships into a space that can be processed by machines, enabling AI to explore and utilize this hidden information through geometric relationships.

And this is precisely the significance of the vector semantic space: it does not make machines truly “see” semantics, but rather builds a bridge connecting human language and mathematical computation.