What an LLM API Is

Table of Contents

1. A Chatbot Is an HTTP Request
2. Set Up the Project
3. Why a Mock Server
4. The Mock LLM Server
5. Start It and Check It Is Alive
6. Your First Request with curl
7. Anatomy of the Request
8. Roles: system, user, assistant
9. Anatomy of the Response
10. Tokens and the usage Field
11. When Things Go Wrong: Status Codes
12. The Test Models
13. Real Providers: Same Idea, Different Names
14. Recipe: The Project So Far
15. Try It Yourself
16. Quick Review: Common Questions

1. A Chatbot Is an HTTP Request

When you type into an AI chat app and press Send, something very ordinary happens. The app sends an HTTP request (the same kind your browser sends to load a web page) to a server. The server runs a large language model and sends back an HTTP response with text inside. That is the whole architecture.

your code  --- request: a list of messages --->  LLM server
your code  <--- response: the next message ----  LLM server

An API (application programming interface) is just a door that a program can knock on. An LLM API is a door that accepts conversations and returns replies. You do not run the model yourself. You do not need a GPU. You send JSON, you get JSON.

In How LLMs Work (for Developers) you looked inside the model: tokens, sampling, and why the model has no memory. This series is the other side. It is about the code that calls a model. Every idea from that series shows up here as a field in a JSON body or a line in a loop.

By the end of this chapter you will have a tiny local server that behaves like an LLM API, you will have talked to it with curl, and you will be able to read every field of a request and a response.

2. Set Up the Project

You need Node.js (the LTS version, 20 or newer) and a terminal. Check that Node is installed:

node --version
# v22.22.2   (yours may differ; 20 or higher is fine)

Create the project folder. We call the toolkit chatlab, and you will add to it in every chapter.

mkdir chatlab
cd chatlab
npm init -y
npm pkg set type=module

npm init -y creates a package.json with default answers. The second command adds "type": "module", which tells Node to treat .js files as ES modules. That is what lets us write import and export and use top-level await, the same style as the earlier series. Open package.json and you will see "type": "module" near the end.

The project uses no dependencies. There is no npm install in this series. Everything is plain Node.js: the built-in fetch and the built-in http module. That keeps the code small and means nothing here will break when a package changes.

Also make two folders for later: mkdir src demo.

3. Why a Mock Server

A real LLM API needs an account, an API key (a secret password that bills your credit card), and an internet connection. That is a poor way to learn. A typo could cost money, a key could leak into a screenshot, and not every student has a card.

So we build the practice on a mock server: a small program on your own computer that accepts the same kind of request and sends back the same kind of response. Be honest about what it is:

• It is not intelligent. Its replies are canned rules. It greets you, repeats what you said, recalls a name from earlier messages, and does a few other tricks, so we can test code against known answers.
• It is free, fast and predictable, ideal while writing code.
• The shape of the conversation (the URL, headers, JSON fields, status codes, streaming format) is the same shape many real providers use. So the code you write against the mock works against a real model after you change two settings. A real model will answer far better, and that is the only big difference you will notice.

4. The Mock LLM Server

Create a file called mock-llm.js in the project root (next to package.json) and paste in this whole program. It has about 90 lines and no dependencies. Read it once now; the sections after it explain each part.

// mock-llm.js — a tiny fake LLM server with a chat-completions-style API.
// Zero dependencies. Run: node mock-llm.js   (default port 8787)
import http from "node:http";

const KEY = "test-key";
let requestCount = 0;

const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
const estimateTokens = (text) => Math.max(1, Math.ceil(text.length / 4));

function makeReply(messages) {
  const users = messages.filter((m) => m.role === "user");
  const last = (users.at(-1)?.content ?? "").toLowerCase();
  const system = messages.find((m) => m.role === "system")?.content ?? "";
  const pirate = system.toLowerCase().includes("pirate");
  const say = (t) => (pirate ? `Arr! ${t}` : t);

  if (/\bjson\b/.test(last)) return '{"answer": "mock json", "confidence": 0.9}';
  if (/what('| i)s my name|who am i/.test(last)) {
    for (const m of [...users].reverse()) {
      const hit = m.content.match(/my name is (\w+)/i);
      if (hit) return say(`Your name is ${hit[1]}.`);
    }
    return say("I do not know your name yet. You have not told me.");
  }
  if (/\b(hello|hi|hey)\b/.test(last)) return say("Hello! How can I help you today?");
  if (/phnom penh/.test(last)) return say("Phnom Penh is the capital of Cambodia.");
  return say(`You said: ${users.at(-1)?.content ?? ""}`);
}

export function startMock(port = 8787) {
  const server = http.createServer(async (req, res) => {
    const send = (status, obj, headers = {}) => {
      res.writeHead(status, { "content-type": "application/json", ...headers });
      res.end(JSON.stringify(obj));
    };
    if (req.method === "GET" && req.url === "/health") return send(200, { ok: true });
    if (req.method !== "POST" || req.url !== "/v1/chat/completions")
      return send(404, { error: { message: "not found" } });

    if (req.headers.authorization !== `Bearer ${KEY}`)
      return send(401, { error: { message: "invalid api key" } });

    let raw = "";
    for await (const chunk of req) raw += chunk;
    let body;
    try { body = JSON.parse(raw); } catch {
      return send(400, { error: { message: "body must be valid JSON" } });
    }
    const { model, messages, stream } = body;
    if (!model || !Array.isArray(messages) || messages.length === 0)
      return send(400, { error: { message: "model and messages are required" } });

    requestCount += 1;
    if (model === "mock-flaky" && requestCount % 3 !== 0)
      return send(429, { error: { message: "rate limit, slow down" } }, { "retry-after": "0" });
    if (model === "mock-error") return send(500, { error: { message: "server exploded" } });
    if (model === "mock-slow") await sleep(3000);

    const text = makeReply(messages);
    const promptTokens = estimateTokens(messages.map((m) => m.content).join(" "));
    const completionTokens = estimateTokens(text);
    const usage = {
      prompt_tokens: promptTokens,
      completion_tokens: completionTokens,
      total_tokens: promptTokens + completionTokens,
    };

    if (!stream) {
      return send(200, {
        id: `mock-${requestCount}`,
        model,
        choices: [{ index: 0, message: { role: "assistant", content: text }, finish_reason: "stop" }],
        usage,
      });
    }

    res.writeHead(200, { "content-type": "text/event-stream", "cache-control": "no-cache" });
    const event = (obj) => res.write(`data: ${JSON.stringify(obj)}\n\n`);
    event({ id: `mock-${requestCount}`, choices: [{ index: 0, delta: { role: "assistant" } }] });
    for (const piece of text.match(/\S+\s*/g) ?? []) {
      await sleep(40);
      event({ id: `mock-${requestCount}`, choices: [{ index: 0, delta: { content: piece } }] });
    }
    event({ id: `mock-${requestCount}`, choices: [{ index: 0, delta: {}, finish_reason: "stop" }], usage });
    res.write("data: [DONE]\n\n");
    res.end();
  });
  return new Promise((resolve) => server.listen(port, () => resolve(server)));
}

if (import.meta.url === `file://${process.argv[1]}`) {
  const port = Number(process.env.PORT ?? 8787);
  await startMock(port);
  console.log(`mock LLM listening on http://localhost:${port}  (api key: ${KEY})`);
}

Here is what the pieces do.

The reply rules. makeReply(messages) looks at the last user message and picks a canned answer. If it contains the word json, you get a small JSON string. If it asks "what is my name" it searches the earlier user messages for "my name is ..." and answers from them. That one rule is how we will prove, in chapter 3, that a model only knows what you send it. Greetings get a greeting, "Phnom Penh" gets a fact, and anything else is echoed back as You said: .... If the system message mentions the word "pirate", every reply starts with Arr! . The reply logic is deliberately silly, because it must be predictable.

The token estimate. estimateTokens guesses one token per four characters. A real provider counts with its model's real tokenizer (see Tokens and Tokenizers), so real numbers differ. The mock only needs numbers of the right shape so you can practise reading and adding up usage.

The routes. The server accepts exactly one real endpoint: POST /v1/chat/completions. A GET /health route exists so you can check the server is alive. Everything else is a 404.

The checks. In order: the API key must be test-key (else 401), the body must be valid JSON (else 400), and it must contain a model and a non-empty messages array (else 400). This is the same order you will meet in real services: who are you, then is your request well-formed.

The response. A normal request returns one JSON object. If the body says "stream": true the server instead sends a series of small events; that is chapter 4. Pay attention to the line requestCount += 1: it makes the response id change on every request.

Standalone mode. The last block runs only when you start the file directly. It reads the port from the PORT environment variable (default 8787). Because the file also exports startMock, other scripts can import it and start the server inside a test.

5. Start It and Check It Is Alive

Open a second terminal (leave your first one for commands). In the project folder run:

node mock-llm.js
# mock LLM listening on http://localhost:8787  (api key: test-key)

The program does not exit. It is a server, waiting for requests. Stop it any time with Ctrl+C. If another program already uses port 8787, pick another one: PORT=8800 node mock-llm.js on macOS or Linux, and use that port in every URL below. (On Windows PowerShell: $env:PORT=8800; node mock-llm.js.)

Now, in your first terminal, knock on the health route:

curl http://localhost:8787/health
# {"ok":true}

curl is a command-line tool that sends HTTP requests. It is installed on macOS, Linux and current Windows. If you see {"ok":true}, the "model" is up.

6. Your First Request with curl

Now ask it something. This one command has everything a chat request needs. The backslashes just split the long command over several lines:

curl http://localhost:8787/v1/chat/completions \
  -H "content-type: application/json" \
  -H "authorization: Bearer test-key" \
  -d '{"model":"mock-1","messages":[{"role":"user","content":"Hello!"}]}'
# {"id":"mock-1","model":"mock-1","choices":[{"index":0,"message":{"role":"assistant","content":"Hello! How can I help you today?"},"finish_reason":"stop"}],"usage":{"prompt_tokens":2,"completion_tokens":8,"total_tokens":10}}

You just called an LLM API. The model's whole answer is inside that JSON: "Hello! How can I help you today?". Everything else is packaging. Because the output is a single line, here it is again, formatted so it is easy to read:

{
  "id": "mock-1",
  "model": "mock-1",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello! How can I help you today?"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 2,
    "completion_tokens": 8,
    "total_tokens": 10
  }
}

The id counts up from mock-1 after each restart, so yours may differ. The reply text, token counts and structure are stable, because the mock has no randomness.

7. Anatomy of the Request

A request has four parts: a method, a URL, some headers, and a body.

Method and URL. POST http://localhost:8787/v1/chat/completions. POST means "here is some data for you to process". The first half, http://localhost:8787, is the base URL (which server). The second half, /v1/chat/completions, is the path (which door). Providers that copy this style use the same path under their own domain, so switching providers is mostly a base-URL change. We will keep these two parts separate in code from the start.

Headers are small labels on the request:

The key is how the server knows who you are and whom to bill. Treat a real key like a password. Chapter 6 is all about keeping it safe.

Body. The JSON you send:

{
  "model": "mock-1",
  "messages": [
    { "role": "user", "content": "Hello!" }
  ]
}

• model says which model should answer. Providers offer several, with different speed, quality and price. You choose by name.
• messages is the whole conversation so far, as an array, oldest first. Each message has a role and some content. This array is the single most important idea in this series, because it is the model's only memory.

Real APIs accept more optional fields. Two you will meet soon are max_tokens (a cap on the reply length) and temperature (how random the choices are; see Probabilities, Temperature and Sampling). The mock ignores them, but chapter 2's client supports both.

8. Roles: system, user, assistant

Every message has a role that says who is speaking. There are three you need now.

Try a system message. Tell the mock to act as a pirate:

curl -s http://localhost:8787/v1/chat/completions \
  -H "content-type: application/json" \
  -H "authorization: Bearer test-key" \
  -d '{"model":"mock-1","messages":[
        {"role":"system","content":"You are a pirate."},
        {"role":"user","content":"Hello!"}]}'
# {"id":"mock-2", ... "content":"Arr! Hello! How can I help you today?" ...
# usage: {"prompt_tokens":6,"completion_tokens":10,"total_tokens":16}

(The output above is trimmed with ... to fit the page; run it to see all of it. The -s flag hides curl's progress meter.) The system message changed the behaviour of the reply. Nothing else in the request changed. A real model follows instructions in the system message far more flexibly than our "does it contain the word pirate" rule, but the mechanism, putting instructions at the front of the messages, is the same. The system message is where you define your assistant.

Now see why assistant messages exist. A chat is a history. Here the user asks something, the assistant replied once, and the user asks a follow-up:

curl -s http://localhost:8787/v1/chat/completions \
  -H "content-type: application/json" \
  -H "authorization: Bearer test-key" \
  -d '{"model":"mock-1","messages":[
        {"role":"user","content":"Hello!"},
        {"role":"assistant","content":"Hello! How can I help you today?"},
        {"role":"user","content":"What is the capital of Cambodia? Phnom Penh?"}]}'
# {"id":"mock-3", ... "content":"Phnom Penh is the capital of Cambodia." ...
# usage: {"prompt_tokens":21,"completion_tokens":10,"total_tokens":31}

You, not the server, wrote that assistant line. The server did not keep it from the earlier call; it has no idea an earlier call happened. You sent a history, and the model continues it. Chapter 3 turns that into a proper Conversation class. For now remember: the client holds the conversation, and every request carries all of it.

A warning for later: a user message is whatever a person typed, and it may contain hostile instructions. Roles help the model decide whom to trust, but they are not a security boundary. Chapter 6 returns to this.

9. Anatomy of the Response

Look again at the response from section 6. Each field has a job:

Why is choices an array? Some APIs can return several alternative replies at once. In practice you almost always ask for one and read choices[0]. Your code to get the answer text will always be the same line:

const text = data.choices[0].message.content;

finish_reason tells you why the text ended. The mock always says "stop", meaning "the model finished naturally". A real API can also say it stopped because it hit your max_tokens limit (often the string "length"), which means the reply was cut off. Good code checks this, because a half-finished answer looks fine until someone reads it. The exact words vary between providers, so check your provider's documentation.

10. Tokens and the usage Field

The last field is usage:

"usage": { "prompt_tokens": 2, "completion_tokens": 8, "total_tokens": 10 }

Remember from Tokens and Tokenizers that a model does not read letters or words. It reads tokens, small pieces of text. The API reports how many tokens each call used:

• prompt_tokens: everything you sent (all messages).
• completion_tokens: everything the model wrote.
• total_tokens: the sum of those two.

This is how you are billed, and it is where the lessons from Context Windows and Memory turn into real money. Providers usually charge separately for input and output tokens, and the input includes the entire history every time. Compare the two pirate requests above. The first sent 2 short messages (6 prompt tokens). The follow-up request sent 3 messages (21 prompt tokens). Each turn re-sends everything, so prompt_tokens keeps growing.

You can check the mock's arithmetic. It estimates one token per four characters, rounded up. For the pirate request, the system text is 17 characters and "Hello!" is 6, joined by a space:

const estimate = (text) => Math.max(1, Math.ceil(text.length / 4));

console.log(estimate("You are a pirate. Hello!"));
// 6   (24 characters / 4)
console.log(estimate("Arr! Hello! How can I help you today?"));
// 10  (37 characters / 4, rounded up)

Both match the usage from section 8. But this is the mock's rule: real tokenizers split text differently, and each model counts its own way. Khmer text often costs more tokens per character than English. Always trust the usage the API reports, not your own guess; use estimates only to plan ahead.

11. When Things Go Wrong: Status Codes

Every HTTP response has a status code, a number that says whether the request worked. 200 means success. Numbers starting with 4 mean you made a mistake; numbers starting with 5 mean the server had a problem. The mock produces three of the most common ones. In the examples below, -s hides curl's progress meter and -w prints the status code.

No key, or the wrong key: 401.

curl -s -w ' HTTP %{http_code}\n' http://localhost:8787/v1/chat/completions \
  -H "content-type: application/json" \
  -H "authorization: Bearer wrong" \
  -d '{"model":"mock-1","messages":[{"role":"user","content":"Hello!"}]}'
# {"error":{"message":"invalid api key"}} HTTP 401

-w ' HTTP %{http_code}\n' asks curl to print the status code after the body. 401 Unauthorized really means "I do not know who you are".

A badly formed request: 400.

curl -s -w ' HTTP %{http_code}\n' http://localhost:8787/v1/chat/completions \
  -H "authorization: Bearer test-key" \
  -d '{"model":"mock-1","messages":[]}'
# {"error":{"message":"model and messages are required"}} HTTP 400

An empty messages array is not a conversation, so the server refuses. Sending something that is not JSON at all (-d 'not json') gives {"error":{"message":"body must be valid JSON"}} HTTP 400.

Wrong door: 404. Asking for any other path, or using GET on the chat path, finds nothing:

curl -s -w ' HTTP %{http_code}\n' http://localhost:8787/v1/chat/completions
# {"error":{"message":"not found"}} HTTP 404

Every error has the same shape: {"error": {"message": "..."}}. Chapter 2's client reads that message and throws a proper JavaScript error. Chapter 7 covers 429 and 500 and the full list, including which ones are worth retrying.

12. The Test Models

The model field normally picks a real model. The mock accepts any name, but gives four names special behaviour, so we can practise failures on demand without waiting for a real outage.

You can see mock-flaky in action right now. Send the same request three times to a freshly started server. The failures are "rate limit" errors; the third one slips through:

for i in 1 2 3; do
  curl -s -w ' HTTP %{http_code}\n' http://localhost:8787/v1/chat/completions \
    -H "authorization: Bearer test-key" \
    -d '{"model":"mock-flaky","messages":[{"role":"user","content":"hi"}]}'
done
# {"error":{"message":"rate limit, slow down"}} HTTP 429
# {"error":{"message":"rate limit, slow down"}} HTTP 429
# {"id":"mock-3","model":"mock-flaky","choices":[...],"usage":{...}} HTTP 200

(Trimmed with .... The pattern depends on the server's request counter, so run it right after a restart.) This is a simple model of a service under load. Real servers fail unpredictably, and your code has to cope. We will.

13. Real Providers: Same Idea, Different Names

Many providers offer a chat endpoint with this same general shape: a POST with a model and a messages array, a Bearer key in a header, and a choices array in the reply. The phrase "OpenAI-compatible" is a widely used name for that request shape, and you will see it in documentation and tools. Treat it as a hint, not a guarantee:

• Some providers use the same shape exactly.
• Some use a different path, a different field for the system prompt, or a different response structure.
• Some options (like max_tokens) may have other names or limits.

So the rule is: always read your provider's documentation. The idea is identical everywhere: send the conversation, get text back. The field names may differ. Because we keep the URL, key, model name and field handling in one small file (chapter 2), adapting to a provider usually means editing that file, not rewriting your app.

This series makes no claims about which model is best, what any provider charges, or what limits apply. Those facts change often; look them up on the provider's own pages.

14. Recipe: The Project So Far

After this chapter your chatlab folder looks like this:

chatlab/
  package.json      ("type": "module")
  mock-llm.js       (the mock server, shown in section 4)
  src/              (empty for now)
  demo/             (empty for now)

Keep two terminals open from now on: one running node mock-llm.js and one for everything else. You will not edit the mock again, and every later chapter assumes it is running. If the port is busy, set PORT for the mock and LLM_BASE_URL for your own code (chapter 2 explains the second one).

15. Try It Yourself

1. Read the server. Without running it, predict the reply and the usage for the message "hey" with no system message. Then run it and compare.
2. Break it on purpose. Send each of these with curl and write down the status code and message: no authorization header, an empty messages array, -d 'hello', a GET request. Which did you expect, and which surprised you?
3. Make it recall. Send a conversation with {"role":"user","content":"My name is Sokha."}, an assistant reply, and then "What is my name?". Then send only the last question. What changed in the reply, and what does that tell you about the server?
4. Watch the bill grow. Send the same user message with 1, 3 and 5 earlier messages in front of it. Write down prompt_tokens each time. Is the growth faster or slower than the number of messages?
5. Pirate or not. Change the system message to "You are a Pirate" (capital P) and then "You are a buccaneer". Read makeReply and explain which one triggers the prefix and why.
6. Count like a tokenizer. Take an English sentence and the same meaning in Khmer. Apply Math.ceil(text.length / 4) to both. Would a real tokenizer give equal numbers? Use Tokens and Tokenizers.

16. Quick Review: Common Questions

Questions beginners often ask about this chapter. Try answering each one yourself first, then click the question to open the answer.

Where to go next. You can now read an LLM API request and response like a book, and you have a server to practise on. Next you will stop using curl and write the code: a small, reusable client with error handling in Your First API Call.