Integers, Overflow and Precision

Table of Contents

1. Numbers Live in Boxes
2. Fixed-Width Integers
3. Overflow: When the Counter Wraps Around
4. JavaScript Numbers and the Safe Integer Limit
5. BigInt: Integers With No Ceiling
6. Floating Point: Why 0.1 + 0.2 Is Strange
7. Comparing Decimals Safely
8. Money Belongs in Integer Cents
9. Rounding Rules
10. Recipe: The money Module
11. Try It Yourself

1. Numbers Live in Boxes

On paper, a number can be as long as you like. Write 9 followed by a hundred more nines and the paper does not complain. A computer is different. Every number has to be stored in a fixed amount of memory, and a fixed amount of memory can only hold a fixed number of different values.

In Numbers and Number Systems you saw that a number is stored as bits. This chapter asks the next question: what happens when the bits run out? The answer explains a famous family of bugs: counters that suddenly go negative, IDs that quietly change by one, and shopping carts that total $3.3000000000000003.

2. Fixed-Width Integers

Many languages (C, Java, Go, Rust, and databases such as MySQL and PostgreSQL) give you integer types with a fixed width, meaning a fixed number of bits. With n bits you get 2^n different values (2^n means 2 multiplied by itself n times).

Unsigned means "zero and up". Signed means "can be negative", and it spends one bit on the sign, which is why the positive side tops out at roughly half the unsigned maximum.

Why is the signed 8-bit range -128 to 127 and not -127 to 127? Because there are exactly 256 patterns, and zero takes one of them. Negative numbers get 128 patterns, positive numbers get 127 plus zero.

You will meet the 32-bit signed limit, 2,147,483,647, constantly. It is the default INT in many SQL databases. If a table's id column is a signed 32-bit INT and your app inserts a couple of million rows a day, the table is full in about three years, and then inserts start failing. That is why large systems use a 64-bit BIGINT for IDs from day one.

3. Overflow: When the Counter Wraps Around

Overflow is what happens when a calculation produces a number too big for its box. In many languages the result wraps around, like a car odometer rolling from 999999 back to 000000.

JavaScript's normal numbers do not wrap (you will see why in the next section), but it has typed arrays that behave like C integers. They are a handy way to see overflow. Create a folder for this chapter's experiments and a file named overflow.js:

const counter = new Int32Array([2147483647]); // biggest signed 32-bit value
counter[0] = counter[0] + 1;
console.log(counter[0]); // -2147483648

const byte = new Uint8Array([255]); // biggest unsigned 8-bit value
byte[0] = byte[0] + 1;
console.log(byte[0]); // 0

const tooBig = new Uint8Array([300]);
console.log(tooBig[0]); // 44

Int32Array stores signed 32-bit integers and Uint8Array stores unsigned 8-bit ones. Adding 1 to the largest value wraps to the smallest. The 300 becomes 44 because 300 - 256 = 44: only the lowest 8 bits survive. That is exactly the "remainder after dividing" idea from Modular Arithmetic.

JavaScript's bitwise operators also force numbers into signed 32-bit form, which is why this trick shows up in older code:

console.log((2147483647 + 1) | 0); // -2147483648
console.log((2 ** 32 + 5) | 0);    // 5

Real-world overflow bugs are not just trivia. A 16-bit counter that wrapped caused scoring glitches in old arcade games, and a 32-bit "seconds since 1970" timestamp will run out in the year 2038 on systems that still store it in a signed 32-bit integer. The fix is always the same idea: use a wider box.

4. JavaScript Numbers and the Safe Integer Limit

In JavaScript, almost every number is a 64-bit floating-point value (the IEEE 754 "double"). It does not wrap at 2 billion. Instead it can represent integers exactly only up to a limit, and beyond that limit it starts to round.

console.log(Number.MAX_SAFE_INTEGER);     // 9007199254740991
console.log(Number.MIN_SAFE_INTEGER);     // -9007199254740991
console.log(Number.MAX_SAFE_INTEGER + 1); // 9007199254740992
console.log(Number.MAX_SAFE_INTEGER + 2); // 9007199254740992
console.log(Number.MAX_SAFE_INTEGER + 1 === Number.MAX_SAFE_INTEGER + 2); // true

That last line is the scary one: two different calculations give the same answer. 9007199254740991 is 2^53 - 1. Above it, the doubles are spaced further apart than 1, so some integers simply do not exist as values and get rounded to a neighbor.

You can ask JavaScript whether an integer is still trustworthy:

console.log(Number.isSafeInteger(2 ** 53 - 1)); // true
console.log(Number.isSafeInteger(2 ** 53));     // false

Why does this matter in practice? Imagine a database with 64-bit IDs such as 9007199254740993. A JavaScript client reads that from JSON as a number and prints 9007199254740992. The ID is now off by one and points at a different record. Many APIs (Twitter/X and Discord are well-known examples) return big IDs as strings for exactly this reason.

Factorials are another easy way to fall off the edge:

let f = 1;
for (let i = 2; i <= 18; i++) f *= i;
console.log(f, Number.isSafeInteger(f)); // 6402373705728000 true

f *= 19;
console.log(f, Number.isSafeInteger(f)); // 121645100408832000 false

18! is the largest factorial that fits safely. At 19! the result may still look plausible, but the last digits can no longer be trusted.

5. BigInt: Integers With No Ceiling

When you need exact whole numbers beyond the safe limit, JavaScript has a second number type: BigInt. Write a BigInt by adding n after the digits, or call BigInt(...).

console.log(9007199254740993);   // 9007199254740992  (wrong by one)
console.log(9007199254740993n);  // 9007199254740993n (exact)
console.log(2n ** 64n);          // 18446744073709551616n
console.log(typeof 10n);         // bigint

BigInts grow as large as memory allows. A few rules to remember:

• You cannot mix BigInt and normal numbers in one expression. 10n + 5 throws TypeError: Cannot mix BigInt and other types, use explicit conversions. Convert one side first: 10n + BigInt(5).
• Division drops the fraction because BigInt has no decimals: 7n / 2n is 3n.
• Number(big) converts back, but you lose exactness if the value is above the safe limit.
• BigInts are slower than normal numbers. Use them when you need them, such as large IDs, huge factorials, or cryptography, not for every loop counter.

Here is the factorial that broke the normal number type, done correctly:

function bigFactorial(n) {
  let result = 1n;
  for (let i = 2n; i <= BigInt(n); i++) {
    result *= i;
  }
  return result;
}

console.log(bigFactorial(20)); // 2432902008176640000n
console.log(bigFactorial(25)); // 15511210043330985984000000n

25! is about 1.55 x 10^25, far beyond Number.MAX_SAFE_INTEGER, and every digit is exact. You will use this in Counting: Permutations and Combinations, where factorials explode quickly.

6. Floating Point: Why 0.1 + 0.2 Is Strange

Integers are the easy case. Decimals are where most surprises live. Open the console and type:

console.log(0.1 + 0.2);         // 0.30000000000000004
console.log(0.1 + 0.2 === 0.3); // false

This is not a JavaScript bug. Python, Java, C, and Go print the same thing. The cause is that computers store numbers in binary, and some decimal fractions cannot be written exactly in binary. It is the same reason 1/3 cannot be written exactly in decimal: you get 0.3333... forever, and at some point you have to cut it off.

In binary, 0.5 and 0.25 are easy (they are 2^-1 and 2^-2), so they are exact. But 0.1 becomes an endless repeating pattern and gets cut off, leaving a tiny error. Add a few of these together and the errors show up:

console.log(0.5 + 0.25);   // 0.75  (exact)
console.log(0.1 * 3);      // 0.30000000000000004
console.log(1.1 + 2.2);    // 3.3000000000000003
console.log(4.35 * 100);   // 434.99999999999994

let total = 0;
for (let i = 0; i < 10; i++) total += 0.1;
console.log(total);        // 0.9999999999999999
console.log(total === 1);  // false

The errors are tiny (around 1 part in a quadrillion) and fine for physics, graphics, or averages. They are not fine for anything where "equal" or "exact to the cent" matters.

7. Comparing Decimals Safely

Since decimal results can be off by a hair, never compare computed decimals with ===. Compare them with a small tolerance instead. This is the nearlyEqual function you wrote in Numbers and Number Systems, and here is why it exists.

import { nearlyEqual } from "./src/numbers.js";

console.log(0.1 + 0.2 === 0.3);          // false
console.log(nearlyEqual(0.1 + 0.2, 0.3)); // true

JavaScript even provides the smallest meaningful gap near 1 as Number.EPSILON:

console.log(Number.EPSILON); // 2.220446049250313e-16
console.log(Math.abs(0.1 + 0.2 - 0.3) < Number.EPSILON); // true

e-16 means "times 10 to the power of -16", so Number.EPSILON is about 0.0000000000000002. Use it for numbers near 1; for much larger or smaller numbers a relative tolerance (what nearlyEqual should do) is better.

Display is a separate problem. toFixed(2) rounds a number to two decimals and returns a string:

console.log((0.1 + 0.2).toFixed(2)); // "0.30"

That is fine for printing, but do not use it to fix up stored values, and as the next sections show, even toFixed has surprises.

8. Money Belongs in Integer Cents

Here is the classic mistake. A shop adds up prices as decimals:

const price = 19.99;
console.log(price * 100);             // 1998.9999999999998
console.log(Math.floor(price * 100)); // 1998

Convert $19.99 to cents by multiplying by 100, drop the fraction, and you have lost a cent. That is the off by one cent bug. It is rare enough to survive testing and common enough that accountants eventually find it.

The fix is a rule used by almost every payment system:

Store and calculate money as whole integers of the smallest unit. Convert to a decimal string only when displaying it.

For US dollars the smallest unit is the cent, so $19.99 is the integer 1999. Integer arithmetic is exact (up to the safe limit, which is about 90 trillion dollars in cents, plenty for a shop):

const itemCents = 1999;
const quantity = 3;
const subtotal = itemCents * quantity;
console.log(subtotal); // 5997

const vat = Math.round((subtotal * 10) / 100); // 10% VAT
console.log(vat);             // 600
console.log(subtotal + vat);  // 6597

Notice that subtotal * 10 happens first with integers, and only the final division produces a fraction, which we round once, deliberately.

What about riel? Cambodian riel (KHR) has no minor unit in everyday use: nobody pays "half a riel". So a KHR amount is already a whole integer, and 41000 means 41,000 riel. At a rate of roughly 4,100 riel per dollar, one US cent is about 41 riel, so converting cents to riel is a plain integer multiplication:

const cents = 1999;
console.log(cents * 41); // 81959 (about 81,959 riel)

Real exchange rates change daily, so treat 41 as an example, and round to a sensible amount (shops often round riel to the nearest 100).

Finally, at the boundary where you show money or send it to another system, convert the integer back to text:

const cents = 1999;
console.log((cents / 100).toFixed(2)); // "19.99"

This is safe because cents is an integer: dividing by 100 and formatting to two decimals will reproduce exactly the digits you stored. Payment gateways usually want exactly this: the ABA PayWay checkout in the ABA PayWay series takes the amount as a string like '1.00', so keep your totals in cents internally and build that string at the very end.

9. Rounding Rules

Sooner or later you divide money (splitting a bill, applying a discount, adding tax) and get a fraction of a cent. You must round, and you should decide how once, in one place.

JavaScript gives you four basic tools:

The surprise is Math.round(-2.5), which is -2, not -3. JavaScript rounds exact halves toward positive infinity.

Always rounding halves up has a slight bias: across millions of transactions, it adds a tiny bit extra. Banker's rounding (also called round half to even) rounds an exact half to the nearest even number, so 0.5 goes to 0, 1.5 goes to 2, 2.5 goes to 2, and 3.5 goes to 4. The rounding errors cancel out on average. Some accounting systems and standards require it, so ask which rule your business uses. Here is a version you can use:

function roundHalfEven(x) {
  const floor = Math.floor(x);
  const diff = x - floor;
  if (diff === 0.5) {
    return floor % 2 === 0 ? floor : floor + 1;
  }
  return Math.round(x);
}

console.log([0.5, 1.5, 2.5, 3.5].map(roundHalfEven)); // [ 0, 2, 2, 4 ]
console.log(roundHalfEven(2.4)); // 2
console.log(roundHalfEven(2.6)); // 3

floor % 2 === 0 asks "is the floor even?", using the remainder operator % from Modular Arithmetic. Be aware this only works well if x is exactly representable; a value like 2.5 is, but values that have already been through decimal arithmetic might not hit exactly .5. That is one more reason to stay in integers for as long as you can.

Now toFixed and Math.round surprises:

console.log(Math.round(1.005 * 100)); // 100  (you might expect 101)
console.log((1.005).toFixed(2));      // "1.00" (you might expect "1.01")
console.log((2.5).toFixed(0));        // "3"
console.log((1.45).toFixed(1));       // "1.4"

The decimal 1.005 is stored as something slightly below 1.005 (1.00499999999999989...), so rounding correctly gives 1.00. The computer is following the math perfectly; it just got a slightly different input than the one you typed. The lesson: parse prices as text into integer cents at the edge, and never round tiny floating-point leftovers back into money.

10. Recipe: The money Module

Time to add this chapter's module to mathlab. Create src/money.js:

// src/money.js

// Convert dollars (a decimal) to whole cents.
export function toCents(dollars) {
  return Math.round(dollars * 100);
}

// Format an integer amount for display.
// USD: amount is in cents.  KHR: amount is whole riel.
export function formatMoney(amount, currency = "USD") {
  if (currency === "KHR") {
    return amount.toLocaleString("en-US") + "៛";
  }
  const sign = amount < 0 ? "-" : "";
  const abs = Math.abs(amount);
  const dollars = Math.trunc(abs / 100);
  const cents = abs % 100;
  return (
    sign + "$" + dollars.toLocaleString("en-US") + "." + String(cents).padStart(2, "0")
  );
}

// Is this number an integer we can still trust?
export function isSafeAmount(n) {
  return Number.isSafeInteger(n);
}

// Add two integer amounts, refusing results that cannot be trusted.
export function addCents(a, b) {
  const sum = a + b;
  if (!Number.isSafeInteger(sum)) {
    throw new RangeError("Amount is outside the safe integer range");
  }
  return sum;
}

// Exact factorial using BigInt.
export function bigFactorial(n) {
  let result = 1n;
  for (let i = 2n; i <= BigInt(n); i++) {
    result *= i;
  }
  return result;
}

// Round half to even ("banker's rounding").
export function roundHalfEven(x) {
  const floor = Math.floor(x);
  const diff = x - floor;
  if (diff === 0.5) {
    return floor % 2 === 0 ? floor : floor + 1;
  }
  return Math.round(x);
}

A few details: padStart(2, "0") pads the cents to two characters so 5 becomes "05". toLocaleString("en-US") adds thousands separators. abs % 100 is the remainder after dividing by 100, which is the cents part.

Now re-export it from src/index.js by adding one line (keep the lines that earlier chapters added):

export * from "./money.js";

Add this to the bottom of demo.js, then run node demo.js:

import {
  toCents, formatMoney, isSafeAmount, addCents, bigFactorial, roundHalfEven,
} from "./src/money.js";

console.log("--- money ---");
console.log(toCents(19.99));                    // 1999
console.log(formatMoney(1999));                 // $19.99
console.log(formatMoney(100000050));            // $1,000,000.50
console.log(formatMoney(-1999));                // -$19.99
console.log(formatMoney(41000, "KHR"));         // 41,000៛
console.log(isSafeAmount(2 ** 53));             // false
console.log(addCents(1999, 501));               // 2500
console.log(bigFactorial(25));                  // 15511210043330985984000000n
console.log([0.5, 1.5, 2.5, 3.5].map(roundHalfEven)); // [ 0, 2, 2, 4 ]

try {
  addCents(Number.MAX_SAFE_INTEGER, 1);
} catch (error) {
  console.log(error.name + ": " + error.message);
  // RangeError: Amount is outside the safe integer range
}

If you already have other imports at the top of demo.js, move this import line up there; ES module imports can sit anywhere at the top level, but tidy code keeps them together.

11. Try It Yourself

1. Find the wrap. Using Int8Array (signed 8-bit), store 127, add 1, and print the result. Predict the answer first, then confirm. What happens with Uint16Array and 65535?
2. Spot the lost ID. Run console.log(9007199254740993) and compare it with console.log(9007199254740993n). Then write a function isIdSafe(id) that returns true only when Number.isSafeInteger(id) is true, and test it with 2 ** 53 - 1 and 2 ** 53.
3. Fix the cart. A cart holds three prices: 1.10, 2.20, and 3.30. Add them as decimals, then convert each to cents with toCents and add the integers. Print both totals and compare. Use formatMoney to print the second one.
4. Add VAT. Write addVat(cents, percent) that returns cents + Math.round((cents * percent) / 100). Test it with 1999 cents and 10 percent. Why do we multiply before dividing?
5. Rounding shootout. For the values 0.5, 1.5, 2.5, -0.5, and -1.5, print Math.round(x) next to roundHalfEven(x). Where do they disagree, and which one has less bias over many values?
6. Biggest safe factorial. Write a loop with normal numbers that multiplies 1 * 2 * 3 * ... and stops just before Number.isSafeInteger becomes false. It should stop at 18. Then compute bigFactorial(30) and count its digits using String(bigFactorial(30)).length.

Next, we'll step back from "how big can a number be" to "how fast does a number grow", with functions, powers, and the logarithm: Functions, Growth and Logarithms.