Vectors: Lists of Numbers

Table of Contents

1. A Vector Is a List of Numbers
2. Set Up the mathai Project
3. Three Pictures of One Idea
4. Add, Subtract, and Scale
5. Length: How Big Is a Vector?
6. Distance: How Far Apart Are Two Vectors?
7. The Catch: One Big Number Shouts Over the Rest
8. Normalize: Keep the Direction, Drop the Size
9. The Mean of Vectors: A Centroid
10. Seeded Random Numbers and Sample Data
11. Where Vectors Appear in AI
12. Recipe: The vector, rng, and data Modules
13. Try It Yourself
14. Quick Review: Common Questions

1. A Vector Is a List of Numbers

Here is the whole idea of this chapter in one sentence:

A vector is a list of numbers, where the position in the list means something.

That is it. If you have written [6, 7, 80] in JavaScript, you have written a vector. The word sounds scary because in physics class it came with arrows and Greek letters. In AI, a vector is just an ordinary array of numbers, and the interesting part is what we do with it.

Why do we care? Because computers can only do arithmetic on numbers, and almost everything in AI (a student, a photo, a song, a sentence) has to become numbers before a program can compare it with anything else. A vector is the standard container for those numbers. Once two things are vectors, you can ask questions like "how far apart are they?" and "which one is closest to this one?" with plain school arithmetic.

This is the first chapter of Math for AI. By the end of the series you will be able to explain, with code you wrote yourself, what an embedding is, what a layer in a neural network does, and what "training" means. Vectors come first because everything else stands on them.

Say it out loud: "A vector is a list of numbers that describes something. Two similar things get similar lists."

2. Set Up the mathai Project

Across the whole series you build one small toolkit called mathai. It is plain Node.js with zero dependencies: no packages to install, no API keys, no network. Make a folder for it. You need Node.js (the LTS version) first.

mkdir mathai && cd mathai
npm init -y
mkdir src

Open the package.json that npm init created and add one line, "type": "module", so Node accepts modern import and export syntax:

{
  "name": "mathai",
  "version": "1.0.0",
  "type": "module"
}

Your npm init -y output has a few more fields (main, scripts, and so on). Leave them alone and just add the "type" line.

The layout you will end up with:

mathai/
  package.json
  src/
    vector.js      <- this chapter
    rng.js         <- this chapter
    data.js        <- this chapter
    similarity.js  <- next chapter
    ...            <- one new module per chapter
  demo-*.js        <- small experiments you create as you read

The files in src/ are the toolkit. They are saved once and imported many times. The demo-*.js files are small throwaway programs that use the toolkit. Each snippet in this series starts with a comment naming its file, so you always know where to put it.

The three files this chapter owns are listed in full in Recipe: The vector, rng, and data Modules. You can save them now and follow along, or build them up as the sections go. Either way, save them before running any demo.

3. Three Pictures of One Idea

A vector is easiest to understand through pictures from real life.

Picture 1: a student. Imagine a teacher who describes each student with three numbers: hours studied per week, hours slept per night, and the last exam score. Ana studies 6 hours, sleeps 7, and scored 80. She is the vector [6, 7, 80]. The first position always means "hours studied", the second always means "hours slept". The position is the label.

Picture 2: a color. Every color on a screen is three numbers: how much red, green and blue, each from 0 to 255. Pure red is [255, 0, 0]. Orange is [255, 165, 0]. A color is a vector with three dimensions.

Picture 3: a word. This one is the AI picture. A program cannot compare the word "coffee" with the word "tea", but it can compare two lists of numbers. So we give each word a list of, say, a few hundred numbers that describe how it is used. Real word vectors are learned by a model, and nobody writes them by hand. For this chapter we invent four numbers just to show the shape.

// demo-pictures.js
const ana = [6, 7, 80]; // hours studied, hours slept, last score
const orange = [255, 165, 0]; // red, green, blue
const coffee = [0.8, 0.1, 0.0, 0.3]; // an invented 4-number "word"

console.log(ana.length, orange.length, coffee.length);
console.log(ana[2]);

// output:
// 3 3 4
// 80

The number of items in a vector is its dimension. Ana and orange have 3 dimensions, our pretend word has 4. Everything in this series works the same whether the dimension is 2 or 2,000. We often draw vectors with two dimensions because paper is flat, but the code does not care.

A picture for two dimensions: each vector is a point on a map, and the two numbers are "how far across" and "how far up".

 hours
 gaming
   10 |  g
      |  g  g
    5 |
      |            c
      |         c  c
    0 +-----------------
      0     5    10   hours coding

Students who code a lot sit on the right, students who game a lot sit at the top. Vectors that are close on the map describe similar students. That one sentence is the seed of search, recommendation, and classification in AI.

4. Add, Subtract, and Scale

Three operations cover most of what we do with vectors.

Add two vectors by adding them position by position. Subtract the same way. Scale a vector by multiplying every number by one ordinary number.

// demo-arithmetic.js
import { add, sub, scale } from "./src/vector.js";

const ana = [6, 7, 80];
const bora = [2, 5, 55];

console.log(add(ana, bora));
console.log(sub(ana, bora));
console.log(scale(ana, 2));
console.log(scale(ana, 0.5));

const step = sub(bora, ana);
console.log(step);
console.log(add(ana, scale(step, 0.5)));

// output:
// [ 8, 12, 135 ]
// [ 4, 2, 25 ]
// [ 12, 14, 160 ]
// [ 3, 3.5, 40 ]
// [ -4, -2, -25 ]
// [ 4, 6, 67.5 ]

Every line is plain school arithmetic. add(ana, bora) is [6+2, 7+5, 80+55]. sub(ana, bora) tells you how Ana differs from Bora: she studies 4 more hours, sleeps 2 more, and scored 25 more.

The last two lines are the most useful trick in the chapter. step is the journey from Ana to Bora. If you walk half of that journey (scale(step, 0.5)) starting at Ana, you land exactly halfway between them: [4, 6, 67.5]. This "start plus a scaled step" move comes back in Gradient Descent and Loss, where a model improves by taking many small steps.

A picture of subtraction and scaling, in 2D:

   B ●
      \
       \   step = B - A  (arrow from A to B)
        ● half way (A + 0.5 * step)
         \
          ● A

Subtraction gives the arrow between two points. Scaling makes the arrow longer or shorter without turning it. A negative scale flips it around.

5. Length: How Big Is a Vector?

How far is the point [3, 4] from the origin [0, 0]? Walk 3 across and 4 up. Pythagoras (the right-angle triangle rule from school) says the straight-line distance is sqrt(3*3 + 4*4), which is 5.

   4 +----● [3, 4]
     |   /
     |  / 5   <- the length (norm)
     | /
   0 +----+
     0    3

The length of a vector is called its norm. For any number of dimensions it is the same recipe: square every number, add them up, take the square root. Our toolkit builds it from the dot product (multiply position by position, then add), which you meet properly in the next chapter: norm(a) = sqrt(dot(a, a)).

// demo-length.js
import { norm, distance, sub } from "./src/vector.js";

console.log(norm([3, 4]));
console.log(distance([0, 0], [3, 4]));
console.log(distance([1, 1], [4, 5]));
console.log(norm(sub([4, 5], [1, 1])));
console.log(norm([1, 2, 2]));
console.log(norm([0, 0]));

// output:
// 5
// 5
// 5
// 5
// 3
// 0

The Pythagoras rule works in 3 dimensions too: [1, 2, 2] has length sqrt(1 + 4 + 4) = 3. In 300 dimensions you cannot draw it, but the arithmetic is the same. Do not worry about "seeing" high dimensions; nobody can. You trust the arithmetic that worked in 2 and 3.

6. Distance: How Far Apart Are Two Vectors?

The distance between two vectors is the length of the arrow between them. Subtract first (that gives the arrow), then take its length:

distance(a, b) = norm(a - b)

Lines 3 and 4 of the last output show it: from [1, 1] to [4, 5] is a 3-4-5 triangle, so the distance is 5. This is called the Euclidean distance, the ordinary "as the crow flies" distance.

Distance answers "how different are these two things?" Small distance means similar, large distance means different. That is already enough to build a classifier, which you will do in the next chapter.

7. The Catch: One Big Number Shouts Over the Rest

Distance has a trap. Let us measure three students. Ana is [6, 7, 80], Bora is [2, 5, 55], and Chenda is [6, 8, 78]:

// demo-dominance.js
import { distance } from "./src/vector.js";

const ana = [6, 7, 80];
const bora = [2, 5, 55];
const chenda = [6, 8, 78];

console.log(distance(ana, bora).toFixed(2));
console.log(distance(ana, chenda).toFixed(2));

// the same two students with every number squeezed into 0..1
const ana01 = [6 / 10, 7 / 10, 80 / 100];
const bora01 = [2 / 10, 5 / 10, 55 / 100];
console.log(distance(ana01, bora01).toFixed(3));

const red = [255, 0, 0];
const orange = [255, 165, 0];
const blue = [0, 0, 255];
console.log(distance(red, orange).toFixed(1));
console.log(distance(red, blue).toFixed(1));

// output:
// 25.40
// 2.24
// 0.512
// 165.0
// 360.6

Ana and Chenda are close (2.24). Ana and Bora are far (25.40). But look at why Ana and Bora are far: the score gap is 25, the study gap is 4, the sleep gap is 2. The exam score is on a bigger scale, so it drowns out the other two numbers. The distance is almost entirely about the score.

When we squeeze every number into the range 0 to 1 (second pair of lines), the picture is fairer: all three numbers matter, and the distance becomes 0.512. That is one example of scaling your features, which the last chapter of this series treats properly in Statistics and the Big Picture.

The colors show the same idea in another way. Red and orange differ only in the green channel (165), while red and blue differ in two channels, so red is much closer to orange than to blue (165 vs about 361). That matches what your eyes say, which is why distance on [r, g, b] is a useful (though rough) idea of "similar color".

The numbers here are small examples we invented. The lesson is general: distance only makes sense when the numbers are on comparable scales.

8. Normalize: Keep the Direction, Drop the Size

Sometimes the size of a vector is not what you care about, only the way it points. Think of two arrows pointing the same way, one short and one long. A unit vector is an arrow of length exactly 1. To normalize a vector, divide every number by its own length:

// demo-normalize.js
import { normalize, norm } from "./src/vector.js";

const u = normalize([3, 4]);
console.log(u);
console.log(norm(u));
console.log(normalize([30, 40]));
console.log(normalize([0, 0]));
console.log(norm(normalize([6, 7, 80])).toFixed(3));

// output:
// [ 0.6, 0.8 ]
// 1
// [ 0.6, 0.8 ]
// [ 0, 0 ]
// 1.000

[3, 4] and [30, 40] have different sizes (5 and 50) but the same direction, so they normalize to the same unit vector [0.6, 0.8]. Any vector except the all-zero one can be normalized, and the result always has length 1.

The all-zero vector has no direction, so you cannot divide by its length (which is 0). Our normalize handles that edge case by returning zeros instead of NaN. This is a small detail that avoids a confusing bug later.

Why does this matter for AI? When you compare two text meanings, a long document and a short sentence about the same topic should count as similar. Throwing away the size and keeping only the direction is exactly what the next chapter's cosine similarity does.

9. The Mean of Vectors: A Centroid

The mean of numbers is easy: add them up, divide by how many. The mean of vectors is the same, done one position at a time. The result is the "average point", called a centroid, the balancing point of a group.

Our students dataset (in data.js) has eight students, each with [hours coding, hours gaming] and a label. Let us find the centre of each group:

// demo-centroid.js
import { mean, distance } from "./src/vector.js";
import { students } from "./src/data.js";

const coders = students.filter((s) => s.label === "coder").map((s) => s.x);
const gamers = students.filter((s) => s.label === "gamer").map((s) => s.x);

const coderCentre = mean(coders);
const gamerCentre = mean(gamers);
console.log(coderCentre);
console.log(gamerCentre);

const newcomer = [6, 4];
console.log(distance(newcomer, coderCentre).toFixed(2));
console.log(distance(newcomer, gamerCentre).toFixed(2));

console.log(mean([[255, 0, 0], [0, 0, 255]]));

// output:
// [ 8.5, 2 ]
// [ 2, 8.5 ]
// 3.20
// 6.02
// [ 127.5, 0, 127.5 ]

The average coder codes 8.5 hours and games 2. The average gamer is the opposite. A new student who codes 6 hours and games 4 is closer to the coder centre (3.20 vs 6.02), so a very simple classifier would call them a coder. That is a real, working idea called nearest centroid.

The last line is a nice surprise: the mean of red and blue is [127.5, 0, 127.5], a purple. Averaging vectors mixes them.

The mean of vectors is everywhere in AI. A common trick is to represent a whole sentence by the average of its word vectors. The sentence vector then sits in the middle of its words. It is simple, it loses the word order, and it still works surprisingly well for rough search.

10. Seeded Random Numbers and Sample Data

Two small modules support every later chapter.

rng.js gives you random numbers that are repeatable. Math.random() is different on every run, which makes tutorials hard to check, because your output would not match the book. mulberry32(seed) is a tiny generator: the same seed always gives the same sequence.

// demo-random.js
import { mulberry32 } from "./src/rng.js";

const rand = mulberry32(42);
console.log(rand().toFixed(4), rand().toFixed(4), rand().toFixed(4));

const again = mulberry32(42);
console.log(again().toFixed(4));

const other = mulberry32(43);
console.log(other().toFixed(4));

// output:
// 0.6011 0.4483 0.8525
// 0.6011
// 0.9998

Seed 42 gives 0.6011 first, every time, on every computer. A new generator with the same seed starts the sequence again. A different seed (43) gives a different sequence. This matters in AI too: a "random" weight start or a random train/test split should be reproducible, or you can never tell whether a change in results came from your idea or from luck. Chapter 8 returns to this.

data.js holds three small datasets that later chapters reuse: study (hours studied against exam score, used to fit a line in the gradient descent chapter), students (the coders and gamers above), and xor (a tiny puzzle a single straight line cannot solve, used in the neural network chapter). All the numbers are invented examples, small enough that you can check results by eye.

11. Where Vectors Appear in AI

Now connect the dots. Almost every idea in modern AI uses vectors:

• Embeddings. A word, a sentence, a photo or a song is turned into a vector of hundreds or thousands of numbers. Similar things get nearby vectors. See Embeddings and Attention for the full story. Real embeddings are learned during training, not written by hand, and a single dimension usually has no human meaning.
• Search by meaning. Turn the question and every document into vectors, then pick the documents whose vectors are nearest the question. That is the core of the RAG tutorials, and you build it in the next chapter.
• Weights. The "knowledge" inside a model is stored as huge tables of numbers, and each row of a table is a vector. Learning means nudging those numbers.

Our toy "word arithmetic" shows the flavor of what people mean by "directions have meaning". With invented axes [royal, male]:

// demo-meaning.js
import { add, sub } from "./src/vector.js";

// axes: [royal, male]. Invented numbers.
const king = [0.9, 0.9];
const man = [0.1, 0.9];
const woman = [0.1, 0.1];
const result = add(sub(king, man), woman);
console.log(result.map((x) => x.toFixed(1)));

// output:
// [ '0.9', '0.1' ]

king - man + woman gives [0.9, 0.1]: very royal, not male, which is "queen". In real learned embeddings, relationships like this sometimes show up approximately, and it is a popular party trick. Treat it as a fun illustration, not a guarantee. Our numbers were chosen by hand to make it work.

Honest caveats. (1) Our vectors have 2 to 4 numbers; real ones have far more. (2) Real embeddings come from training, not from a person choosing axes. (3) A vector is only a representation. Two vectors being close means the model's numbers are close, which usually, but not always, tracks meaning in a way humans find sensible. We do not say a vector "understands" anything.

One more gotcha, about our own code. Our add assumes both vectors have the same length, and JavaScript will not warn you if they do not:

// demo-mismatch.js
import { add } from "./src/vector.js";

console.log(add([1, 2, 3], [10, 20]));
console.log(add([1, 2], [10, 20, 30]));

// output:
// [ 11, 22, NaN ]
// [ 11, 22 ]

The first call hits a missing number and gives NaN ("not a number"). The second silently drops the extra number. Neither crashes. When your AI code gives odd results, a length mismatch between vectors is one of the first things to check. In Matrices and Layers you will see the same problem as a "shape mismatch" error.

12. Recipe: The vector, rng, and data Modules

Save these three files in the src folder of your mathai project.

src/vector.js:

// Vectors are plain arrays of numbers.
export const add = (a, b) => a.map((x, i) => x + b[i]);
export const sub = (a, b) => a.map((x, i) => x - b[i]);
export const scale = (a, k) => a.map((x) => x * k);
export const dot = (a, b) => a.reduce((sum, x, i) => sum + x * b[i], 0);
export const norm = (a) => Math.sqrt(dot(a, a));
export const distance = (a, b) => norm(sub(a, b));

export function normalize(a) {
  const n = norm(a);
  return n === 0 ? a.map(() => 0) : a.map((x) => x / n);
}

export function cosine(a, b) {
  const na = norm(a);
  const nb = norm(b);
  return na === 0 || nb === 0 ? 0 : dot(a, b) / (na * nb);
}

// Average of many vectors, component by component.
export function mean(vectors) {
  const sum = vectors.reduce((acc, v) => add(acc, v));
  return scale(sum, 1 / vectors.length);
}

src/rng.js:

// Seeded random numbers so every run prints the same output.
export function mulberry32(seed) {
  let a = seed >>> 0;
  return function () {
    a = (a + 0x6d2b79f5) >>> 0;
    let t = a;
    t = Math.imul(t ^ (t >>> 15), t | 1);
    t ^= t + Math.imul(t ^ (t >>> 7), t | 61);
    return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
  };
}

src/data.js:

// Small original datasets used across the series.
// [hours studied per week, exam score out of 100]
export const study = [
  [1, 48], [2, 52], [3, 60], [4, 63], [5, 71], [6, 74],
  [7, 80], [8, 83], [9, 88], [10, 93],
];

// [hours coding per week, hours gaming per week] with a label
export const students = [
  { x: [9, 2], label: "coder" }, { x: [8, 3], label: "coder" },
  { x: [10, 1], label: "coder" }, { x: [7, 2], label: "coder" },
  { x: [2, 9], label: "gamer" }, { x: [3, 8], label: "gamer" },
  { x: [1, 10], label: "gamer" }, { x: [2, 7], label: "gamer" },
];

// The classic XOR problem: a single straight line cannot solve it.
export const xor = [
  { x: [0, 0], y: 0 }, { x: [0, 1], y: 1 },
  { x: [1, 0], y: 1 }, { x: [1, 1], y: 0 },
];

To check the wiring, create demo-check.js, import add and mean from ./src/vector.js and study from ./src/data.js, and print add([1, 2], [3, 4]) (you should see [ 4, 6 ]), mean([[1, 2], [3, 4], [5, 9]]) ([ 3, 5 ]) and study.length (10).

Every output in this chapter is stable: the same numbers on every run and every machine, because nothing here depends on time or on Math.random(). The toolkit also exports dot and cosine; you will use them in the next chapter.

13. Try It Yourself

1. Your own vector. Describe yourself or a friend with four numbers (hours of sleep, hours of study, hours of sport, hours of screens). Write two such vectors and print their distance. Then change the units of one number (hours to minutes) and print the distance again. Why did the answer change so much?
2. Halfway. Use add, sub and scale to compute the point one-quarter of the way from [0, 0] to [8, 4]. You should see [ 2, 1 ].
3. Unit check. Normalize five different vectors of your choice and print norm of each result. They should all print 1 (maybe with a tiny float error such as 0.9999999999999999; that is normal).
4. Centroid of three. Use mean on the three vectors [1, 1], [2, 5] and [9, 0]. Then compute it by hand and compare.
5. Classify the newcomer. Using the students data, pick a newcomer [5, 5] and decide whether the nearest centroid is the coder or the gamer one. What does the tie-like result tell you about the limits of this simple method?
6. Seeds. Make two generators with mulberry32(7). Print the first three numbers from each. Are they equal? Now draw one extra number from the first before printing its three. What happens?

14. Quick Review: Common Questions

Questions beginners often ask about this chapter. Try answering each one yourself first, then click the question to open the answer.

Where to go next. You can now store things as vectors and measure how big, how far apart, and how central they are. Distance is one way to ask "how alike?". The other is to ask whether two vectors point the same way, and that is the dot product, the most important operation in the series: Dot Product and Similarity.