1. How does AI compare vectors?

In the previous article (see:AI Semantic Space Exploration (Part 2): From a Word to an Article—How Semantic Vectors Are GeneratedWe already know that whether it's a single word, a sentence, or an entire article, it can all be transformed into semantic vectors through the Embedding model and enter the same semantic space.

However, vector generation is not the end point. Once different content is converted into vectors of a uniform form, a new problem arises:How should AI determine whether these contents are semantically similar?

For humans, we can rely on experience and intuition to judge whether two pieces of content are similar; but for AI, it needs to use mathematical methods to analyze the feature differences between vectors in order to determine their proximity in semantic space.

This problem may seem like a simple vector comparison, but in reality, it exists in almost all AI applications that rely on semantic understanding. For example, when clicking on the "Related Articles" recommendation on my blog's article interface, why does the system recommend related topics based on the article content, instead of simply recommending articles from the same category? Why can semantic search find content relevant to the meaning of the question? And why can RAG (Retrieval Augmentation) find the most suitable text for reference from a large amount of data?

While these capabilities have different applications, they all rely on the same fundamental process: converting content into vectors, and then comparing the differences between these vectors to determine if they have similar semantic expressions. So, what method should be used to compare the differences between vectors?

2. When comparing vectors, what exactly are we comparing?

In order to compare vectors, we need to first clarify a fundamental question:When we say "compare two vectors", what exactly are we comparing? To understand this, let's first re-examine the concept of "vector" itself.

In the previous article, we learned that Embedding transforms a piece of text into a semantic vector. However, many people still think of a vector as just a point—as if it were merely a coordinate in a semantic space. Mathematically, however, a vector is typically represented as...An arrow that starts from the originIt not only records the position of the arrow tip, but also includes information such as length and direction.

As shown in the diagram above: position indicates the coordinates of the arrow's endpoint in space, that is, where it lands; length indicates how long the arrow is, that is, its size; and direction indicates where the arrow is pointing.

Since a vector contains so much information, there can't possibly be only one way to "compare two vectors".

For example, if we are concerned withLocationThe most direct method is to calculate the distance between them. However, this distance is not the distance between the two arrows as a whole, but rather...The distance in space between the two arrow tips (i.e., the endpoints of the vectors)The closer the arrow tips are, the closer the two vectors are in space; the farther apart the arrow tips are, the farther apart they are. This is precisely the kind of comparison we are most familiar with in the real world.

If what we care about islengthTherefore, the comparison involves whether the sizes of the two vectors are similar. Some vectors may point in the same direction but have vastly different lengths; others may have very similar lengths but point in completely different directions.

And if what we care about isdirectionThen the focus shifts again. We are now more concerned with whether the two arrows are pointing in similar directions, rather than simply their lengths or the distance between their tips. Therefore,Different comparison methods essentially involve selecting different information from the vectors as the basis for judgment.

Returning to the question posed in Chapter 1, what we truly want to address is not "whether two vectors are close in geometric space," but rather "whether two contents are semantically similar." Since semantics have already been mapped to vector space, the key question is: among these observable geometric features, which relationship best reflects the degree of semantic closeness?

3. Why does AI often focus on vector direction when judging semantic similarity?

In the previous chapter, we learned that a vector does not simply represent a point in semantic space. It simultaneously contains multiple observable pieces of information: the location of the arrowhead, the length of the vector itself, and the direction it points. These pieces of information are not inherently superior or inferior; different pieces of information actually correspond to different questions.

So, when our goal is to determine whether two pieces of content are semantically similar, how should we understand the meaning of each of these pieces of information? To answer this question, we need to first observe what information position, length, and direction can provide in vector comparison.

First, let's look at the positional relationships.

If our goal is to analyze the overall distribution of a large amount of content in the semantic space, then the position of vectors becomes important. For example, in my blog, articles introducing Cloudflare Tunnel, Cloudflare Access, and Cloudflare Zero Trust, because they all revolve around Cloudflare-related technologies, might have their corresponding vectors forming a relatively concentrated region in the semantic space. Topics such as home network architecture and database optimization, on the other hand, might be distributed in different locations.

In this context, AI focuses on "which content is grouped together in the semantic space," so positional relationships can help complete tasks such as topic classification and automatic clustering.

However, when the task becomes determining whether two pieces of content are semantically similar, one cannot draw a conclusion solely based on the fact that "the original distance is closer." This is because whether distance in vector space can accurately reflect semantic relationships depends on the objective adopted by the embedding model during training and what information the task aims to retain.

Therefore, at least in the semantic similarity scenarios discussed in this paper, the original distance between vectors is usually not suitable as the primary criterion for judgment.

Next, let's look at the length.

Vector length can sometimes carry additional features calculated by the model. For ease of understanding, it can be temporarily regarded as the "overall size" of the content: two pieces of content may have different lengths even if they discuss the same topic, due to differences in expression, scope of information, etc.

For example, consider two articles about Cloudflare Tunnel. One simply introduces what it is and what problems it solves; the other not only introduces the basic concepts but also explains its working principles, network processes, configuration methods, and practical deployment experience. Due to the differences in their scope and scale of information, the length representation of their corresponding vectors may also differ.

However, when it comes to judging semantic similarity, the amount of content is not the most crucial issue. A more detailed article doesn't necessarily mean it discusses something entirely different from another. Therefore, while length may reflect certain expressive features, it cannot directly represent the degree of semantic similarity.

So, the remaining question is: if we want to determine whether the core meanings of two pieces of content are similar, which type of information in the vector is more relevant to this goal? The answer is direction. Compared to position and length, direction is closer to the issue that semantic similarity judgment focuses on. This is because, in the semantic space formed by embeddings, vector direction reflects the semantic expression trend of the content after model transformation.

For example, a short article introducing home servers and an article discussing home data center architecture in detail may contain different amounts of information and have different scopes of expression, but if they revolve around similar issues, their semantic orientation may still remain similar.

However, this raises a new question: while direction is the information we want to compare, length still exists in every vector. So, when two vectors have different lengths, will this difference affect the direction comparison? If so, how should AI eliminate this effect?

4. Why is normalization needed? How does normalization standardize the length?

In the previous chapter, we learned that for semantic similarity judgment, we prefer to compare the directional relationship between vectors. However, the problem is that direction is not isolated; each vector also contains length information. Therefore, before comparing directions, we need to solve the problem of how to eliminate the influence of length differences.

To understand this problem, let's start with a more intuitive proportional relationship. Suppose we compare the body structures of two people:

A's height is 180cm and weight is 72kg.
B's height is 150cm and her weight is 60kg.

If we only look at the absolute values, it's easy to assume that A is larger in every dimension. However, if we focus on "whether the structures are similar," then what truly matters is no longer the total amount, but the proportional relationship between different dimensions.

Calculations show that:

A: 72 / 180 = 0.4
B: 60 / 150 = 0.4

The two are in exactly the same proportion. This indicates that although their overall size is different, their internal structural relationships have not changed; they are simply in different scaling states.

The same logic applies when applied to vector spaces. For example, consider two two-dimensional vectors:

A = (2, 1)
B = (4, 2)

It can be seen that B is simply an enlarged version of A. The component ratios of the two vectors remain consistent.

A: 2 / 1 = 2
B: 4 / 2 = 2

Therefore, although they are different in length, the directional relationship they express has not changed.

The problem thus becomes clear: if different vectors differ only in their overall scaling, then when comparing them, this scaling difference needs to be eliminated first; otherwise, the length factor may affect the comparison results.

This is precisely the problem that vector normalization solves. The core idea of normalization is simple: for any vector, first calculate its own length, then use this length as a scaling reference to proportionally adjust the entire vector so that all vectors eventually have the same length (usually uniformly 1).

Formally, it can be expressed as:

v' = v / |v|

For two-dimensional vectors:

(x', y') = (x / √(x² + y²), y / √(x² + y²))

It's important to note that normalization doesn't adjust each dimension individually; rather, it scales the entire vector uniformly. Therefore, although the specific values of each dimension change, the proportional relationship between them remains constant.

In other words, normalization transforms the vector representation, which originally included both "direction" and "length," into a form more suitable for direction comparison. Once all vectors are within the same length standard, their semantic similarity can be further determined by measuring the degree of directional proximity.

5. How do we measure whether the directions of two vectors are close?

After normalization, the length difference between vectors has been eliminated. Therefore, the only problem left to solve is: how to measure whether the directions of two vectors are close.

Returning to the arrow analogy, all arrows are now of the same length. Therefore, when comparing two vectors, what we really need to observe is no longer which is longer, but the angle between them. The smaller the angle, the closer the two vectors point; the larger the angle, the more pronounced the difference in their directions.

In mathematics, a classic method for measuring how close the directions of two vectors are is to calculate the cosine of the angle between them. This is the reason for the existence of cosine similarity.

Cosine similarity doesn't directly calculate the distance between two vectors. Instead, it uses the cosine of the angle between them to convert the "proximity of direction" into a comparable numerical value. When two vectors are in the same direction, the angle between them is 0°, and the cosine value is 1, indicating they are highly similar in direction. When two vectors are in opposite directions, the angle is 180°, and the cosine value is -1. And when two vectors are perpendicular, the angle is 90°, and the cosine value is 0.

Therefore, cosine similarity has actually completed a transformation: it has transformed the originally difficult-to-measure "degree of directional similarity" into a clear mathematical indicator.

In other words, once semantics are mapped to a vector space, the question of "whether two contents are similar" is essentially redefined as a geometric problem: whether the directions in which two vectors point in space are close. Cosine similarity is the mathematical tool used to answer this question.

Some readers may have a new question here: Since we already know that we are comparing the angle between two vectors, why do we actually use the "cosine value" instead of directly calculating the angle itself?

The reason is that directly processing angles is not the most convenient way for computers, while cosine similarity can reflect the directional relationship between two vectors and has the advantages of simple calculation and numerical stability. Therefore, cosine similarity has become a commonly used method for measuring vector similarity in vector retrieval, semantic search, and most embedding applications.

It's important to note that cosine similarity itself isn't an AI-specific algorithm; it's simply a mathematical tool in linear algebra used to describe the directional relationships between vectors. The key to its application in semantic understanding lies in the fact that embedding transforms text into vectors, and normalization eliminates length differences, allowing this calculation method, originally belonging to geometric space, to be naturally used to measure the closeness between semantic terms.

6. Postscript: Vector similarity is merely a projection of semantic understanding.

In the preceding content, we deduced from "how semantics is represented" to "how vectors are compared," ultimately arriving at the seemingly simple calculation method of cosine similarity. At this point, a complete chain has been established: text is converted into vectors, vectors have position, length, and direction in space, and semantic similarity is transformed into a judgment of the degree of directional proximity.

However, stopping there can easily lead to a misunderstanding: that semantic understanding itself is simply geometric operations in vector space. In fact, vector space is merely one way we observe and process semantics; it's more like a "projection" of complex semantic relationships.

The concept of "projection" can be understood as follows: complex semantic phenomena are compressed into a mathematical space, and then the relationships within are approximated using geometric concepts such as position, distance, angle, and direction. This process is very efficient in engineering, but it is not equivalent to semantics itself; rather, it is a computable mode of expression.

Therefore, the "similarity" between vectors is essentially just a mathematical mapping of semantic relations in the vector space. Cosine similarity has become the most common method in semantic comparison because it corresponds to a core assumption established earlier: in many semantic similarity tasks, we are more concerned with whether the directions of two content expressions are close, rather than their absolute positions or scale differences in space.

However, it's important to note that cosine similarity is not the only reasonable comparison method. Different tasks focus on different information: some scenarios prioritize the distance between vectors, while others aim to preserve the information carried by length. Therefore, the specific comparison method used still depends on the model and the task objective.

Returning to the original question: How does AI determine whether two vectors are similar? Now we can put it more accurately: AI does not directly understand the concept of "similarity," but rather, within a designed semantic space, it chooses an appropriate mathematical method to approximate the relationship between the content.

Therefore, vector space is not semantics itself, but rather a computable representation of semantics in the mathematical world. It maps complex and difficult-to-describe semantic relationships into a space that can be processed by machines, allowing AI to explore and utilize this hidden information through geometric relationships.

This is precisely the significance of vector semantic space: it does not allow machines to truly "see" semantics, but rather establishes a bridge connecting human language and mathematical computation.