Errors, Retries and Cost

Table of Contents

1. Things Will Go Wrong
2. Reading Errors: The Status Code Table
3. Retry Only What Can Succeed
4. Backoff, Jitter and retry-after
5. Testing Retry Without Waiting
6. Timeouts
7. Cost: Tokens Times Price
8. A Hard Budget Stop and max_tokens
9. The Duplicate-Charge Caution
10. Logging Without Leaking
11. Friendly Messages for People
12. Recipe: The retry, budget and errors Modules
13. Try It Yourself
14. Quick Review: Common Questions

1. Things Will Go Wrong

So far your chatbot has lived a happy life. The mock server is on your own computer, answers fast, and never has a bad day. A real API call travels over Wi-Fi or mobile data, into a queue of other people's requests, through a model that does heavy math, and back. Any step can fail: the network drops, the provider says "slow down", a short outage happens, a request hangs, you send a bad key, or your bill grows faster than you planned.

A beginner's chatbot crashes with an ugly stack trace on the first of these. A good chatbot recovers quietly from problems that fix themselves, stops when the problem is real, tells the person something kind, and never spends more than you allowed.

This chapter adds three small modules to chatlab: src/retry.js (try again the right way), src/budget.js (count tokens and money, stop at a limit) and src/errors.js (friendly messages and safe logs).

The mock server has three misbehaving models for exactly this. A real provider will not fail on demand, but the mock will:

Start the mock (node mock-llm.js) in another terminal. Some outputs below depend on how many requests the mock has already seen, because mock-flaky counts all requests. I say so where it matters, and the outputs shown are from a fresh mock.

2. Reading Errors: The Status Code Table

The first clue is the HTTP status code. You met ApiError in chapter 2: it carries status, the server's message, and retryAfter when the server sent that header. The code tells you whose fault it is and whether trying again can help.

Providers differ. Some use 400 for an oversized prompt, and some add their own codes, so read your provider's error page. Also, 429 can mean "you are going too fast" (retry helps) or "your plan is used up" (retry never helps). The error message usually tells you which. The rule still holds:

• 4xx except 429: your request is wrong. Retrying gives the same error forever. Fix it.
• 429 and 5xx: possibly temporary. Retry, politely.

See both kinds against a fresh mock:

// errors-demo.js
import { chat } from "./src/client.js";

const messages = [{ role: "user", content: "hi" }];

for (const model of ["mock-flaky", "mock-flaky", "mock-flaky", "mock-error"]) {
  try {
    const r = await chat({ messages, model });
    console.log(model, "ok:", r.text);
  } catch (e) {
    console.log(model, e.name, e.status, e.message, "retryAfter:", e.retryAfter);
  }
}
// output
// mock-flaky ApiError 429 rate limit, slow down retryAfter: 0
// mock-flaky ApiError 429 rate limit, slow down retryAfter: 0
// mock-flaky ok: Hello! How can I help you today?
// mock-error ApiError 500 server exploded retryAfter: null

All four lines are stable on a fresh mock. The third call worked because it was the 3rd request the mock had seen. On a mock that has served other requests the pattern shifts, because the counter is shared. Real flakiness has no tidy schedule either.

3. Retry Only What Can Succeed

A retry means "send the same request again". It is a great tool and a dangerous one if used blindly. A wrong key retried three times gives three 401 errors. A loop that retries with no limit and no pause can hammer the provider and get your account flagged. The rules:

1. Retry only temporary failures: 429, 5xx, network errors, timeouts. Never 400, 401, 403, 404.
2. Limit the tries. About 3, then give up and tell the person.
3. Wait between tries, longer each time.
4. Obey the server if it says how long to wait.

Start with "should we retry this error?" as its own function. Create src/retry.js:

// src/retry.js (part 1)
import { ApiError } from "./client.js";

// Which failures are worth trying again? Only the ones that can pass later.
export function isRetryable(err) {
  if (err instanceof ApiError) return err.status === 429 || err.status >= 500;
  // fetch() rejects with a TypeError when the network fails;
  // AbortSignal.timeout() rejects with a TimeoutError.
  return err?.name === "TypeError" || err?.name === "TimeoutError";
}

Two families of errors exist. If the server answered with any status, fetch does not throw, and your chat() turns a bad status into an ApiError. If no answer arrived (no network, server down), fetch throws a TypeError with the message fetch failed. Try a port where nothing listens:

try {
  await fetch("http://localhost:8839/");
} catch (err) {
  console.log(err.name, err.message, "| cause:", err.cause?.code);
}
// output
// TypeError fetch failed | cause: ECONNREFUSED

The name and message are stable. The cause.code is the operating system's reason and can differ elsewhere (ENOTFOUND for an unknown host name, ECONNRESET when a connection is cut midway).

One error you must not retry is AbortError, which you get when your own code cancels a request (the Stop button in the next chapter). The user asked to stop. isRetryable returns false for it because its name is neither TypeError nor TimeoutError.

4. Backoff, Jitter and retry-after

How long should you wait? Think of a busy restaurant phone line. If you redial 100 times a second, you do not get through faster, and you block the line for everyone. The standard answer has two ideas:

• Exponential backoff. Wait a little, then double: 0.5 s, 1 s, 2 s, 4 s. An overloaded server gets more and more room to recover. A cap (say 8 s) stops the wait growing forever.
• Jitter. Add randomness. If 1,000 apps got a 503 at the same moment, without jitter they all retry together at 0.5 s, then again at 1 s, hitting the server in waves. With full jitter each app waits a random time between 0 and the current backoff value, and the wave spreads out.

And a third idea: when a server says 429 it often sends a retry-after header with how many seconds to wait. That beats any guess of yours, because the server knows when it will be ready. Our readError already parses it into err.retryAfter. Add the loop to src/retry.js:

// src/retry.js (part 2)
const realSleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

// Run fn(attempt) until it works, or give up after `retries` extra tries.
// `random` and `sleep` can be replaced, which makes tests stable.
export async function withRetry(fn, options = {}) {
  const {
    retries = 3,
    baseMs = 500,
    maxMs = 8000,
    random = Math.random,
    sleep = realSleep,
    onRetry = () => {},
  } = options;

  for (let attempt = 0; ; attempt++) {
    try {
      return await fn(attempt);
    } catch (err) {
      if (attempt >= retries || !isRetryable(err)) throw err;
      // The server's own advice wins. Otherwise: exponential backoff
      // (base, 2x, 4x, ...) with "full jitter" (a random share of it).
      const backoff = Math.min(maxMs, baseMs * 2 ** attempt);
      const delayMs =
        err instanceof ApiError && err.retryAfter !== null
          ? err.retryAfter * 1000
          : Math.round(random() * backoff);
      onRetry({ attempt: attempt + 1, delayMs, error: err });
      await sleep(delayMs);
    }
  }
}

It is only 25 lines:

• fn is a function that makes the call, not the call itself. We must run it again, so we hand over a recipe, not a result.
• If it throws, we check: out of retries, or an error that should not be retried? Then we rethrow the original error, so callers still see the real status.
• Otherwise we compute a delay, tell onRetry (handy for logs), sleep, and loop.
• random and sleep are injectable. Real code uses the defaults. Tests replace them, so output is stable and nothing really waits.

Now the real thing: mock-flaky fails twice with 429, then works.

// retry-demo.js
import { chat } from "./src/client.js";
import { withRetry } from "./src/retry.js";

const messages = [{ role: "user", content: "hi" }];

const reply = await withRetry(
  () => chat({ messages, model: "mock-flaky" }),
  {
    retries: 3,
    baseMs: 10,
    onRetry: ({ attempt, delayMs, error }) =>
      console.log(`retry ${attempt} in ${delayMs}ms because ${error.status}`),
  },
);
console.log(reply.text);
// output (fresh mock server)
// retry 1 in 0ms because 429
// retry 2 in 0ms because 429
// Hello! How can I help you today?

All three lines are stable on a fresh mock. The delay is 0ms, not a jittered number near 10, because the mock sends retry-after: 0 and the server's advice wins.

Now a wrong key. Run this with LLM_API_KEY=wrong so config.js picks up a bad key:

// bad-key-demo.js
import { chat } from "./src/client.js";
import { withRetry } from "./src/retry.js";

let tries = 0;
try {
  await withRetry(() => { tries++; return chat({ messages: [{ role: "user", content: "hi" }] }); },
    { baseMs: 10 });
} catch (err) {
  console.log(`${err.status} ${err.message} after ${tries} try`);
}
// LLM_API_KEY=wrong node bad-key-demo.js
// output
// 401 invalid api key after 1 try

One try, no waiting, a clear error. Compare mock-error, which is always 500: we retry up to the limit, then give up. Here the jitter is made repeatable with a tiny seeded random function so you can compare numbers:

import { chat } from "./src/client.js";
import { withRetry } from "./src/retry.js";

// A tiny seeded random number generator, so the jitter is the same
// on every run. (Real code should use the default Math.random.)
let seed = 12345;
const random = () => ((seed = (seed * 16807) % 2147483647) - 1) / 2147483646;

try {
  await withRetry(
    () => chat({ messages: [{ role: "user", content: "hi" }], model: "mock-error" }),
    { retries: 3, baseMs: 100, random, sleep: async () => {},
      onRetry: ({ attempt, delayMs, error }) =>
        console.log(`  retry ${attempt} in ${delayMs}ms (${error.status})`) },
  );
} catch (err) {
  console.log("gave up:", err.status, err.message);
}
// output
//   retry 1 in 10ms (500)
//   retry 2 in 167ms (500)
//   retry 3 in 379ms (500)
// gave up: 500 server exploded

All stable, because the generator is seeded. The waits are random shares of 100, 200 and 400 ms. After three retries we rethrow the real error with its real status, so the caller can decide what to tell the user.

5. Testing Retry Without Waiting

Because fn, random and sleep are injectable, you can test withRetry with no server and no waiting:

import { withRetry } from "./src/retry.js";
import { ApiError } from "./src/client.js";

let calls = 0;
const flaky = async () => {
  calls++;
  if (calls < 3) throw new ApiError(500, "boom");
  return "done";
};

const delays = [];
const result = await withRetry(flaky, {
  retries: 5,
  baseMs: 100,
  random: () => 1,                      // always the maximum jitter
  sleep: async (ms) => delays.push(ms), // do not really wait
});
console.log(result, calls, delays);

calls = 0;
try {
  await withRetry(async () => { calls++; throw new ApiError(400, "bad"); },
    { sleep: async () => {} });
} catch (e) {
  console.log("400 ->", e.status, "calls:", calls);
}

// retry-after: 2 seconds beats the backoff formula.
const waits = [];
calls = 0;
await withRetry(async () => {
  calls++;
  if (calls === 1) throw new ApiError(429, "slow down", 2);
  return "ok";
}, { sleep: async (ms) => waits.push(ms) });
console.log("retry-after 2 ->", waits);
// output
// done 3 [ 100, 200 ]
// 400 -> 400 calls: 1
// retry-after 2 -> [ 2000 ]

Everything is stable. The function ran 3 times and slept 100 then 200 ms (doubling). A 400 ran exactly once. The server's 2 seconds became 2000 ms. These checks guard against the two classic bugs: retrying too much, and retrying what can never work.

A note on retry-after: the HTTP standard also allows a date instead of seconds. readError uses Number(...), so a date becomes NaN. Many AI providers send seconds. If yours sends a date, handle it (Exercise 5). And if a server wants you to wait 10 minutes, do not hold a chat request open that long. Choose a maximum you accept, and fail with a clear message beyond it.

6. Timeouts

A timeout answers "how long am I willing to wait?" Without one, a request that hangs hangs forever: your user stares at a spinner and your server holds a connection open for nothing.

Node gives you AbortSignal.timeout(ms), a signal that fires after ms milliseconds. Pass it as the signal option that chat() already supports. Test with mock-slow (3 seconds) and a half-second limit:

// timeout-demo.js
import { chat } from "./src/client.js";
import { withRetry } from "./src/retry.js";

const messages = [{ role: "user", content: "hi" }];

const t0 = Date.now();
try {
  await withRetry(
    () => chat({ messages, model: "mock-slow", signal: AbortSignal.timeout(500) }),
    { retries: 1, baseMs: 10, onRetry: ({ attempt, error }) =>
        console.log(`retry ${attempt} after ${error.name}`) },
  );
} catch (err) {
  console.log("final:", err.name, "-", err.message);
}
console.log("took about", Math.round((Date.now() - t0) / 100) / 10, "seconds");
// output
// retry 1 after TimeoutError
// final: TimeoutError - The operation was aborted due to timeout
// took about 1 seconds

The first two lines are stable. The time is about one second (two tries of 0.5 s plus a short wait) and may print 1 or 1.1.

Create the signal inside the function you retry. A signal fires once and stays fired, so one created outside would be already expired on the second attempt. Each attempt needs a fresh timer.

How long should a timeout be? There is no single number. A long non-streaming answer can take many seconds. With streaming (chapter 4) you care about the time to the first token and about silence in the middle. Measure your own app and add a margin. Also think about total patience: three tries with a 20-second timeout can mean over a minute of waiting. For an interactive chat, one retry and a clear error are often better than a minute of silence.

7. Cost: Tokens Times Price

Now the money. Every response includes usage: tokens in (prompt_tokens) and tokens out (completion_tokens). You met these in chapter 1 and in tokens. Hosted models usually charge per million tokens, with one price for input and a (usually higher) one for output:

cost = (input_tokens × input_price + output_tokens × output_price) / 1,000,000

Create src/budget.js. The prices below are made-up EXAMPLES. They are not any provider's real price. Real prices change, differ by model, and must come from your provider's price page.

// src/budget.js (part 1)

// EXAMPLE prices only. Look up your provider's real price list.
export const examplePrices = { inputPerMillion: 1.0, outputPerMillion: 3.0 };

export function estimateCost(usage, { inputPerMillion, outputPerMillion }) {
  return (
    (usage.prompt_tokens * inputPerMillion +
      usage.completion_tokens * outputPerMillion) /
    1_000_000
  );
}

A chat that sent 12,000 tokens of history and got 800 back:

import { estimateCost, examplePrices } from "./src/budget.js";

const usage = { prompt_tokens: 12000, completion_tokens: 800 };
console.log(estimateCost(usage, examplePrices));
// the same by hand: 12000 * 1 / 1e6 + 800 * 3 / 1e6
console.log((12000 * 1) / 1e6 + (800 * 3) / 1e6);
// output
// 0.0144
// 0.0144

Both lines are the same and stable. With the example prices that is about 1.4 US cents. Cheap alone, but chapter 3 showed that history is resent every turn, so the input side grows with each message. Fifty such turns cost fifty times as much, and a public bot with a thousand users can burn real money overnight. Small per call, large in total: that is why you need a counter and a limit.

Two honest cautions. Estimates are not invoices: cached input at another price, tools, images and extra reasoning tokens can change the total, so check the provider's dashboard for the truth. And the mock's tokens are fake: it divides characters by four, while a real provider uses a real tokenizer.

8. A Hard Budget Stop and max_tokens

A budget is a promise: "this program will not spend more than X". Enforce it at three levels:

1. In your code (here): count usage and refuse new calls past a limit.
2. Per user (chapters 6 and 8): each visitor gets a daily allowance.
3. At the provider: set a monthly spending limit or alert in their dashboard. This is your safety net if your own code has a bug. Always do this one.

Add the counter to src/budget.js:

// src/budget.js (part 2)
export class BudgetError extends Error {
  constructor(message) {
    super(message);
    this.name = "BudgetError";
  }
}

// Adds up usage and refuses new calls once the limit is reached.
export class Budget {
  constructor({ maxTokens = Infinity, maxCost = Infinity, prices = examplePrices } = {}) {
    this.maxTokens = maxTokens;
    this.maxCost = maxCost;
    this.prices = prices;
    this.promptTokens = 0;
    this.completionTokens = 0;
    this.calls = 0;
  }
  get totalTokens() { return this.promptTokens + this.completionTokens; }
  get cost() {
    return estimateCost(
      { prompt_tokens: this.promptTokens, completion_tokens: this.completionTokens },
      this.prices,
    );
  }
  // Call this after every successful response.
  add(usage) {
    if (!usage) return;
    this.calls += 1;
    this.promptTokens += usage.prompt_tokens;
    this.completionTokens += usage.completion_tokens;
  }
  // Call this BEFORE every request.
  check() {
    if (this.totalTokens >= this.maxTokens)
      throw new BudgetError(`token budget used up (${this.totalTokens}/${this.maxTokens})`);
    if (this.cost >= this.maxCost)
      throw new BudgetError(`cost budget used up ($${this.cost.toFixed(6)}/$${this.maxCost})`);
  }
}

The pattern is "check before, add after". A tiny limit (30 tokens) shows the stop quickly:

// budget-demo.js
import { chat } from "./src/client.js";
import { Budget } from "./src/budget.js";

const budget = new Budget({ maxTokens: 30, maxCost: 1 });
const messages = [{ role: "user", content: "hello there" }];

for (let i = 1; i <= 5; i++) {
  try {
    budget.check();
    const { usage } = await chat({ messages });
    budget.add(usage);
    console.log(`call ${i}: total tokens ${budget.totalTokens}, cost $${budget.cost.toFixed(6)}`);
  } catch (err) {
    console.log(`call ${i}: ${err.name}: ${err.message}`);
    break;
  }
}
// output
// call 1: total tokens 11, cost $0.000027
// call 2: total tokens 22, cost $0.000054
// call 3: total tokens 33, cost $0.000081
// call 4: BudgetError: token budget used up (33/30)

All stable, since the mock's token counts come from text length. Call 3 went over the limit (33 is more than 30): the check runs before a call and cannot know how big the answer will be. A budget stops you after you cross a line, not exactly at it, so leave a margin. The BudgetError is thrown before any request, so nothing is sent and nothing is spent. It is not an ApiError, so withRetry never repeats it.

To limit one call's size, use max_tokens. Most chat APIs accept a field like it (the name can differ) that sets the most tokens the model may produce. Your client.js supports it as maxTokens. We can see what goes on the wire:

import { requestInit } from "./src/client.js";

const init = requestInit({ messages: [{ role: "user", content: "hi" }], maxTokens: 50 });
console.log(init.body);
// output
// {"model":"mock-1","messages":[{"role":"user","content":"hi"}],"max_tokens":50}

Two honest points. The mock ignores max_tokens: we can check the field is sent, not that it is obeyed. A real model stops at the limit and may cut a sentence in half; the response then usually reports a different stop reason (often finish_reason: "length" instead of "stop"), which you should check. And it is a ceiling, not a target: it cuts the answer off, it does not make it shorter on purpose. To get shorter answers, also ask for them in the system prompt.

Use all the controls together: a short max_tokens, trimmed history (chapter 3), a budget in code, and a spending limit at the provider.

9. The Duplicate-Charge Caution

A subtle trap with retries: you send a request and the timeout fires. Did it fail? You do not know. Maybe the server never got it. Or maybe it did all the work and the answer was lost on the way back, and the provider already billed the tokens. If you retry, you pay twice for one answer.

For a chat message the damage is a few extra tokens. But the same idea is serious for anything with a side effect, like charging a card or sending an email. Ways to stay safe:

• Retry read-like calls freely. Asking a question has no side effect on your side.
• Count a timed-out call as possibly spent. Budget only adds usage when a response arrives, so timed-out calls are invisible to it. The provider dashboard is the final authority.
• Use idempotency keys when the API offers them. An idempotency key is a unique ID you send with a request. If the server sees the same key twice, it returns the first result instead of repeating the work. Check whether your provider supports it, and do not assume.
• Take extra care when tools act in the real world. If your bot later sends messages or changes data, design retries for that deliberately.
• Log the attempt count. A request that retried twice may have run three times.

10. Logging Without Leaking

When something breaks at night, logs are your only witness. They are also where secrets and private data leak: copied to dashboards, shared in chat, and read by many people long after the request is over.

Log facts about the call, not its content. Safe and useful: time, model name, duration, token counts, status code, attempt number, a request ID. Never log the API key or the Authorization header, full request headers, and, by default, the messages. Chat messages can hold names, phone numbers, health worries and passwords typed by mistake. If you really need content to debug, keep it briefly, on purpose, and tell your users.

Create src/errors.js with a safe log line:

// src/errors.js (part 1)

// What YOU keep in the logs: facts about the call, never the content.
export function logLine({ model, ms, usage, error }) {
  return JSON.stringify({
    model,
    ms,
    promptTokens: usage?.prompt_tokens,
    completionTokens: usage?.completion_tokens,
    error: error ? `${error.name}${error.status ? " " + error.status : ""}` : undefined,
  });
}

Notice what is missing on purpose: no messages, no headers, no key, and no error.message (some providers echo part of your input in their error text). One JSON object per line is easy to search later. Do not log the whole error object either: in some libraries it carries the original request, headers and key included. Pick the fields you need.

11. Friendly Messages for People

A person using your chatbot should never see ApiError 429 or a stack trace, or provider text that confuses them or reveals your setup. Map each failure to a short, kind sentence that says what to do next. Add the second part of src/errors.js (the import lines go at the top of the file; the recipe below shows the final order):

// src/errors.js (part 2)
import { ApiError } from "./client.js";
import { BudgetError } from "./budget.js";

// What the PERSON sees. No status codes, no provider text, no keys.
export function userMessage(err) {
  if (err instanceof BudgetError) return "This chat has reached its limit. Please start a new one later.";
  if (err.name === "TimeoutError") return "The answer took too long. Please try again.";
  if (err instanceof ApiError) {
    if (err.status === 429) return "Too many requests right now. Please wait a moment and try again.";
    if (err.status >= 500) return "The AI service is having a problem. Please try again soon.";
    return "Something went wrong on our side. We are looking into it.";
  }
  return "Could not reach the AI service. Check your connection and try again.";
}

The 4xx message is vague on purpose. A 401 or 400 is your bug and the person cannot fix it. Show a calm message, and put the real status in the log. Never show "invalid API key" to the public: it tells attackers which door to try.

Now put it all together: budget check, retries, timeout, logging and a friendly message around one function:

// safe-chat-demo.js
import { chat } from "./src/client.js";
import { withRetry } from "./src/retry.js";
import { Budget } from "./src/budget.js";
import { userMessage, logLine } from "./src/errors.js";

const budget = new Budget({ maxTokens: 500 });

export async function safeChat(messages, model = "mock-1") {
  const started = Date.now();
  try {
    budget.check();
    const result = await withRetry(
      () => chat({ messages, model, maxTokens: 200, signal: AbortSignal.timeout(2000) }),
      { retries: 3, baseMs: 10 },
    );
    budget.add(result.usage);
    console.log("log:", logLine({ model, ms: Date.now() - started, usage: result.usage }));
    return result.text;
  } catch (err) {
    console.log("log:", logLine({ model, ms: Date.now() - started, error: err }));
    return userMessage(err);
  }
}

const q = [{ role: "user", content: "hi" }];
console.log("user sees:", await safeChat(q));
console.log("user sees:", await safeChat(q, "mock-flaky"));
console.log("user sees:", await safeChat(q, "mock-error"));
// output (fresh mock server; the "ms" numbers vary)
// log: {"model":"mock-1","ms":28,"promptTokens":1,"completionTokens":8}
// user sees: Hello! How can I help you today?
// log: {"model":"mock-flaky","ms":5,"promptTokens":1,"completionTokens":8}
// user sees: Hello! How can I help you today?
// log: {"model":"mock-error","ms":43,"error":"ApiError 500"}
// user sees: The AI service is having a problem. Please try again soon.

The user sees: lines and everything in the log: lines except ms are stable. The person never saw the 429 errors that were retried silently, or the three retries of mock-error, and every attempt left a clean log line with no content and no key. With LLM_API_KEY=x you get a 401: the log says "error":"ApiError 401" and the person sees only "Something went wrong on our side." One try, no retry, the real reason in the log.

One more design point: in a streaming chatbot an error can happen after words were already shown, and you cannot un-print them. Show the partial text, add a small "the reply was interrupted" note, and let the user retry. Retrying automatically mid-stream would repeat the beginning. Chapter 8 handles this in the browser.

12. Recipe: The retry, budget and errors Modules

Three new files in chatlab, all wrapped around the old modules. You did not change client.js, stream.js or conversation.js, so the reliability layer can be added or removed without touching the code that talks to the API.

• src/retry.js is part 1 (section 3) followed by part 2 (section 4), exactly as shown, 41 lines in all.
• src/budget.js is part 1 (section 7) followed by part 2 (section 8), 50 lines.
• src/errors.js is the two imports, then userMessage (section 11), then logLine (section 10), 25 lines. Here it is in final order:

// src/errors.js
import { ApiError } from "./client.js";
import { BudgetError } from "./budget.js";

// What the PERSON sees. No status codes, no provider text, no keys.
export function userMessage(err) {
  if (err instanceof BudgetError) return "This chat has reached its limit. Please start a new one later.";
  if (err.name === "TimeoutError") return "The answer took too long. Please try again.";
  if (err instanceof ApiError) {
    if (err.status === 429) return "Too many requests right now. Please wait a moment and try again.";
    if (err.status >= 500) return "The AI service is having a problem. Please try again soon.";
    return "Something went wrong on our side. We are looking into it.";
  }
  return "Could not reach the AI service. Check your connection and try again.";
}

// What YOU keep in the logs: facts about the call, never the content.
export function logLine({ model, ms, usage, error }) {
  return JSON.stringify({
    model,
    ms,
    promptTokens: usage?.prompt_tokens,
    completionTokens: usage?.completion_tokens,
    error: error ? `${error.name}${error.status ? " " + error.status : ""}` : undefined,
  });
}

Remember the limits: everything is single-process and in memory. A Budget resets when the program restarts, and each server instance counts alone. A real service keeps counters in a database or cache. For learning and small bots, this is a sound start.

13. Try It Yourself

1. Retry count. Change retries to 0 in retry-demo.js and run it on a fresh mock. Then try 1 and 2. For which values does mock-flaky succeed, and why?
2. Make it ugly, then fix it. Write a loop that retries forever with no delay against mock-error. Run it for one second and count the requests. What would a real provider think if a thousand users ran it?
3. Streaming and retries. Wrap chatStream (chapter 4) in withRetry and run it on mock-flaky, printing tokens as they arrive. What goes wrong if a stream fails after printing some words? What rule would stop a retry repeating output the user has seen?
4. A total deadline. Add a deadlineMs option to withRetry: if the next wait would pass the deadline, give up now. Test it with random: () => 1 and a fake sleep.
5. Parse a date. Make readError handle a retry-after header that is an HTTP date (such as Wed, 21 Oct 2026 07:28:00 GMT) by turning it into seconds from now, with a sensible maximum. Test it with a fake header.
6. Per-user budgets. Make a Map from user ID to Budget, 200 tokens each. Show that user A hitting the limit does not stop user B. How would you reset all budgets every day?

14. Quick Review: Common Questions

Questions beginners often ask about this chapter. Try answering each one yourself first, then click the question to open the answer.

Where to go next. Your chatbot now survives bad networks and cannot spend without limit. The last step is to let other people use it. The next chapter, A Web Chat UI, builds a browser page that talks to your backend proxy from API Keys and a Backend Proxy, streams the reply word by word, and ends with a checklist for sharing it safely.