A Command-Line Chatbot

Table of Contents

1. What We Are Building
2. Setup: Start the Mock Server
3. Reading Lines with readline
4. The Smallest Chatbot
5. Testing Without a Keyboard
6. Slash Commands
7. Counting Tokens Across the Session
8. When a Request Fails
9. Ctrl+C Stops the Reply, Not the Chat
10. Keeping the History Bounded
11. Recipe: The cli Module
12. A Real Test for the Chatbot
13. Try It Yourself
14. Quick Review: Common Questions

1. What We Are Building

Four chapters of pieces are now sitting in your chatlab folder:

• src/config.js reads the base URL, key and model from the environment (chapter 2).
• src/client.js makes one request and turns failures into ApiError.
• src/conversation.js keeps the message array (chapter 3).
• src/stream.js reads a reply token by token (chapter 4).

None of them is a chatbot yet. They are parts on a workbench. In this chapter you write one more file, src/cli.js, that connects them into a program you can run and talk to:

you> hi
bot> Hello! How can I help you today?

Why a terminal and not a web page? Because a terminal chatbot has almost no extra machinery. There is no HTML, no browser, no server. That makes it the best place to learn the loop that every chatbot runs:

1. read a line from the user,
2. add it to the conversation,
3. send the whole conversation to the model,
4. print the reply as it streams in,
5. add the reply to the conversation,
6. go back to step 1.

A web chat in chapter 8 runs the same six steps. Only step 1 and step 4 change (a form instead of a keyboard, a <div> instead of the terminal).

One honest reminder: the model on the other end is still the mock server, so its answers are canned. We are testing our code, and the code is what a real provider would drive too.

2. Setup: Start the Mock Server

In one terminal, start the mock server from chapter 1. It stays running and prints one line:

node mock-llm.js
# mock LLM listening on http://localhost:8787  (api key: test-key)

Open a second terminal in the same chatlab folder for everything else. If port 8787 is busy on your computer, start the mock on another port and tell your code where it is:

PORT=8820 node mock-llm.js
LLM_BASE_URL=http://localhost:8820 node src/cli.js

The examples below used port 8820, so you will see localhost:8820 in some output (yours shows 8787 by default). LLM_BASE_URL comes from src/config.js, which is why the code never changes.

3. Reading Lines with readline

Node has a built-in module, node:readline, that turns a stream of characters into lines. You create an interface on process.stdin (the keyboard, or a pipe) and loop over it with for await:

// echo.js
import readline from "node:readline";

const rl = readline.createInterface({ input: process.stdin });
process.stdout.write("you> ");
for await (const line of rl) {
  console.log(`(got ${line.length} characters)`);
  process.stdout.write("you> ");
}
console.log("\ninput closed");

Run it and type two lines (press Enter after each, then Ctrl+D to end the input). To keep this page readable, here the same two lines are piped in instead of typed:

printf 'hello\nsawasdei\n' | node echo.js
# you> (got 5 characters)
# you> (got 8 characters)
# you> 
# input closed

Two things to notice. The loop ends by itself when the input ends: Ctrl+D at a keyboard, or the end of a pipe or file. And process.stdout.write prints without adding a newline, which console.log always does. We need that for the you> prompt, and later for streaming tokens onto the same line.

Why for await and not rl.on("line", ...)? Because of order. The event style calls your handler for every line immediately, without waiting for the previous call to finish. With slow network calls inside, replies can overlap and finish in the wrong order. I tested a handler where later lines finish sooner: three lines in as one, two, three came out as three, two, one. With for await, the loop body (every await inside it too) finishes before the next line is read. One question at a time, which is how a conversation should work.

4. The Smallest Chatbot

Now the first real version. It is only the six steps from section 1:

// v0.js
import readline from "node:readline";
import { Conversation } from "./src/conversation.js";
import { chatStream } from "./src/stream.js";

const conversation = new Conversation();
const rl = readline.createInterface({ input: process.stdin });

for await (const line of rl) {
  conversation.user(line);
  process.stdout.write("bot> ");
  const { text } = await chatStream({ messages: conversation.messages() }, (piece) =>
    process.stdout.write(piece)
  );
  process.stdout.write("\n");
  conversation.assistant(text);
}

Read it slowly. conversation.user(line) adds the question. We pass conversation.messages() (system prompt plus every turn so far) to chatStream. The second argument is a callback that receives each piece of text the moment it arrives, and process.stdout.write(piece) prints it with no newline, so the words appear one after another. When the stream ends, text holds the full reply, and we save it with conversation.assistant(text). That last line is the memory. Skip it and the model would never see its own earlier answers.

Test it by telling it your name and asking for it back:

printf 'my name is Sok\nwhat is my name?\n' | node v0.js
# bot> You said: my name is Sok
# bot> Your name is Sok.

The mock's second answer is correct only because we resent the first message. That is the stateless lesson from chapter 3, now running inside a loop. The first reply, "You said: ...", shows what the mock does with anything it does not recognize: it echoes your words back. A real model would say something useful. Both lines here are stable, because the mock has no randomness.

5. Testing Without a Keyboard

Typing the same questions every time you change the code gets tiring, and it is not repeatable. The trick used above is worth stating clearly: your program reads from standard input, so a pipe or a file can stand in for the keyboard.

printf 'hi\n/usage\n/quit\n' > script.txt
node src/cli.js < script.txt

Every example in this chapter is run that way, so what you see is the real output. This works because the streaming delays inside the mock change when characters appear, never which characters. The final text is the same on every run.

There is one wrinkle. At a real terminal, the terminal itself shows what you type. With piped input nothing shows it, so the transcript would look like you> bot> ... with your questions missing. The final cli.js fixes this with one line, using process.stdin.isTTY (true when a human is at a terminal, false for a pipe or file):

if (!process.stdin.isTTY) console.log(line); // a pipe is not echoed by a terminal

The output then reads like a transcript, and a chat can be saved as a script and replayed later (section 12).

6. Slash Commands

A chatbot needs a way to talk to the program, not to the model. Typing "start over" cannot work, because the model would treat it as a question. The usual solution is a command prefix: lines that start with / are for the program and never reach the model.

Our commands:

The code is one switch (full file in section 11). line.slice(1) drops the slash, split(" ") separates the command name from its argument, and default catches typos so that /rest does not get sent to the model as a question. Here is a session that uses most of them, including the pirate persona from chapter 3:

printf 'hi\nmy name is Sok\nwhat is my name?\n/system Talk like a pirate.\nwhat is my name?\n/reset\nwhat is my name?\n/nope\n/quit\n' | node src/cli.js
# chatlab chat  (mock-1 at http://localhost:8820)  type /help
# you> hi
# bot> Hello! How can I help you today?
# you> my name is Sok
# bot> You said: my name is Sok
# you> what is my name?
# bot> Your name is Sok.
# you> /system Talk like a pirate.
# (system prompt updated)
# you> what is my name?
# bot> Arr! Your name is Sok.
# you> /reset
# (conversation cleared)
# you> what is my name?
# bot> Arr! I do not know your name yet. You have not told me.
# you> /nope
# unknown command: /nope  (try /help)
# you> /quit

Follow the story. Before /system, the bot knows the name. After /system, it still knows the name, because changing the system prompt does not touch the history, and now it talks like a pirate. After /reset the name is gone, but the pirate system prompt stays, because reset() clears only the turns. The Arr! on the last answer proves it.

All lines here are stable. The first line shows your own base URL and model, so yours may differ.

Should /system also reset the chat? We keep the history so you can see both behaviours. Exercise 3 asks you to change it.

7. Counting Tokens Across the Session

In chapter 1 you saw that every reply carries a usage object, and chapter 4 showed it arriving in the last streamed event. A chatbot should keep a running total, because tokens are what you pay for. After every finished reply, the code adds usage.prompt_tokens and usage.completion_tokens to a tally object.

The if (usage) guard in the code matters. A provider might not send usage on a streamed reply unless you ask for it, and field names differ between providers. Your code should keep working when it is missing. Run a short chat, type /usage, and look at the last lines:

printf 'hi\nmy name is Sok\nwhat is my name?\n/usage\n/quit\n' | node src/cli.js | tail -5
# you> /usage
# (3 requests, 59 tokens in, 19 tokens out; 6 messages kept)
# you> /quit
#
# bye. 3 requests, 59 tokens in, 19 tokens out

Look at "59 tokens in" next to "19 tokens out". The model wrote only 19 tokens, but we sent three times as many. Why? Because request 2 resent the first exchange, and request 3 resent both. This is the cost growth from Context Windows and Memory showing up in real numbers from your own program.

The mock estimates tokens as characters divided by four, so the exact numbers are the mock's, not a real tokenizer's. The pattern (input grows faster than output) is the same with a real API. On exit, the program prints the totals one last time so you never leave a session without knowing what it consumed.

8. When a Request Fails

The network is not reliable and keys expire. The model server can be down, the key can be wrong, you can hit a rate limit. A chatbot that crashes with a stack trace on the first hiccup is no fun to use. A good one says what happened and lets you try again.

The catch is in the conversation. Our loop adds your message before the call. If the call then fails and we do nothing, your question stays in the history with no answer. The next request would contain two user messages in a row, and the model sees a confusing conversation. So on failure we remove the question:

try {
  const { text: answer, usage } = await chatStream(/* ... */);
  conversation.assistant(answer);
  // ... update the tally
} catch (err) {
  conversation.turns.pop(); // the question was never answered
  const detail = err instanceof ApiError ? `API error ${err.status}: ${err.message}` : err.message;
  console.log(`(failed: ${detail}. Your message was not kept; try again.)`);
}

ApiError is the class from chapter 2. It has a status and a readable message. Anything else (like the network being down) is a plain Error whose message we print as it is. Try three kinds of failure. The first sends a wrong key, the second uses the always-broken mock-error model, and the third points at a port where nothing is running:

printf 'hi\n/usage\n' | LLM_API_KEY=wrong node src/cli.js
# chatlab chat  (mock-1 at http://localhost:8820)  type /help
# you> hi
# bot> 
# (failed: API error 401: invalid api key. Your message was not kept; try again.)
# you> /usage
# (0 requests, 0 tokens in, 0 tokens out; 0 messages kept)
# you> 
# bye. 0 requests, 0 tokens in, 0 tokens out

printf 'hi\n' | LLM_MODEL=mock-error node src/cli.js | grep failed
# (failed: API error 500: server exploded. Your message was not kept; try again.)

printf 'hi\n' | LLM_BASE_URL=http://localhost:8829 node src/cli.js | grep failed
# (failed: fetch failed. Your message was not kept; try again.)

In the first run the program keeps going and /usage still works, and 0 messages kept proves the failed question was removed. The message fetch failed is what Node's fetch reports when it cannot connect at all. The last example assumes nothing is listening on port 8829. It is a network error rather than an HTTP error, so there is no status code.

We only report errors here. We do not retry. A 401 should never be retried, but a 500 or a rate limit often should. That logic gets its own chapter: Errors, Retries and Cost.

9. Ctrl+C Stops the Reply, Not the Chat

Real models can write for a long time. Sometimes you see in the first sentence that the answer is going the wrong way, and you want to stop it. Chapter 4 gave us the tool: an AbortController. We create one per reply and pass its signal into chatStream through the options.

(In the code: current = new AbortController(), and signal: current.signal in the options.)

Then we catch the Ctrl+C key press. In Node, Ctrl+C at a terminal raises a SIGINT signal. By default that kills the process. We install our own handler with two behaviours.

While a reply is streaming, Ctrl+C cancels the request. When you are just waiting at the you> prompt, Ctrl+C quits, which is what people expect. In the catch block we recognise the cancellation by its name, AbortError, print (stopped), and keep what the user already saw.

Notice partial. The onToken callback also collects the text in a variable as it goes, so even if chatStream throws, we still hold the part that was printed. We save that partial answer in the history. The user has seen it and may say "continue", and the model should see that too. If nothing had arrived yet, we drop the question like any failure.

To prove it works, here is a small script that starts the chatbot, sends a long question, presses "Ctrl+C" (by sending SIGINT to the child process) a quarter of a second later, then keeps chatting:

// sigint-demo.js
import { spawn } from "node:child_process";

const p = spawn("node", ["src/cli.js"], { stdio: ["pipe", "inherit", "inherit"] });
p.stdin.write("tell me a long long story about many many words in a row today\n");
setTimeout(() => process.kill(p.pid, "SIGINT"), 250);
setTimeout(() => {
  p.stdin.write("hi\n/usage\n");
  p.stdin.end();
}, 700);

node sigint-demo.js
# chatlab chat  (mock-1 at http://localhost:8820)  type /help
# you> tell me a long long story about many many words in a row today
# bot> You said: tell me a 
# (stopped)
# you> hi
# bot> Hello! How can I help you today?
# you> /usage
# (1 requests, 29 tokens in, 8 tokens out; 4 messages kept)
# you> 
# bye. 1 requests, 29 tokens in, 8 tokens out

The reply was cut after "You said: tell me a" and the chat carried on. The mock sends one word every 40 milliseconds, so after 250 ms you get only the first few words. How many words appear depends on timing, so yours may stop one word earlier or later. That is the part that varies. Everything else is stable.

Look at the usage line. It says 1 requests, not 2, because the stopped reply never delivered its final usage event. That is honest accounting: the tally only counts replies that finished. Be aware that with a real provider, a cancelled request may still be billed for the tokens it produced before the cancel. The tally is a floor, not a guarantee. The provider's dashboard is the final word.

Four messages are kept: the long question, the partial answer, "hi", and its reply.

10. Keeping the History Bounded

A chat that runs for hours would send a longer and longer array, and eventually fail or cost a lot. Chapter 3 gave Conversation a trim() method. We call it every time we add a question, with CHAT_MAX_TURNS (default 20) as the limit. trim(20) keeps the newest 20 messages (10 question and answer pairs). The system prompt is held separately, so it is never trimmed. The limit is read from an environment variable so you can test it with a tiny number:

printf 'one\ntwo\nthree\n/usage\n' | CHAT_MAX_TURNS=3 node src/cli.js
# chatlab chat  (mock-1 at http://localhost:8820)  type /help
# you> one
# bot> You said: one
# you> two
# bot> You said: two
# you> three
# bot> You said: three
# you> /usage
# (3 requests, 34 tokens in, 12 tokens out; 4 messages kept)
# you> 
# bye. 3 requests, 34 tokens in, 12 tokens out

Six messages were exchanged (three questions, three answers), but only 4 are kept. Before sending the third question, trim(3) cut the history to the 3 newest messages (including that new question), and the reply made it 4.

This simple trimming has a flaw, and it is worth knowing. Cutting by message count can leave an assistant message at the front, with its question cut off. Some providers reject a conversation that does not start with a user message, and others accept it but the result reads oddly. Exercise 5 asks you to fix it. The memory chapter described smarter options (token budgets, summaries) if you need them.

11. Recipe: The cli Module

Here is the whole file, src/cli.js. Every piece from the sections above is in it. Read it from top to bottom once. You should recognise every line.

// src/cli.js — a command-line chatbot.  Run: node src/cli.js
import readline from "node:readline";
import { Conversation } from "./conversation.js";
import { chatStream } from "./stream.js";
import { ApiError } from "./client.js";
import { baseUrl, defaultModel } from "./config.js";

const MAX_TURNS = Number(process.env.CHAT_MAX_TURNS ?? 20);
const conversation = new Conversation();
const tally = { requests: 0, promptTokens: 0, completionTokens: 0 };
let current = null; // AbortController of the reply being written, if any

const HELP = `Commands:
  /reset          forget the conversation (keeps the system prompt)
  /system <text>  change the system prompt
  /usage          show tokens used in this session
  /help           show this help
  /quit           leave (Ctrl+D works too)
Press Ctrl+C while a reply is streaming to stop it.`;

function usageLine() {
  const { requests, promptTokens, completionTokens } = tally;
  return `${requests} requests, ${promptTokens} tokens in, ${completionTokens} tokens out`;
}

// Returns true when the line was a command (even an unknown one).
function runCommand(line) {
  const [name, ...rest] = line.slice(1).split(" ");
  const arg = rest.join(" ").trim();
  switch (name) {
    case "reset":
      conversation.reset();
      console.log("(conversation cleared)");
      break;
    case "system":
      if (!arg) console.log(`system prompt: ${conversation.system}`);
      else {
        conversation.system = arg;
        console.log("(system prompt updated)");
      }
      break;
    case "usage":
      console.log(`(${usageLine()}; ${conversation.turns.length} messages kept)`);
      break;
    case "help":
      console.log(HELP);
      break;
    case "quit":
    case "exit":
      return "quit";
    default:
      console.log(`unknown command: /${name}  (try /help)`);
  }
  return "ok";
}

async function reply(text) {
  conversation.user(text).trim(MAX_TURNS);
  current = new AbortController();
  let partial = "";
  process.stdout.write("bot> ");
  try {
    const { text: answer, usage } = await chatStream(
      { messages: conversation.messages(), signal: current.signal },
      (piece) => { partial += piece; process.stdout.write(piece); }
    );
    process.stdout.write("\n");
    conversation.assistant(answer);
    tally.requests += 1;
    if (usage) {
      tally.promptTokens += usage.prompt_tokens;
      tally.completionTokens += usage.completion_tokens;
    }
  } catch (err) {
    process.stdout.write("\n");
    if (err.name === "AbortError") {
      console.log("(stopped)");
      // Keep what the user already saw, so the next turn makes sense.
      if (partial) conversation.assistant(partial.trim());
      else conversation.turns.pop();
    } else {
      conversation.turns.pop(); // the question was never answered
      const detail = err instanceof ApiError ? `API error ${err.status}: ${err.message}` : err.message;
      console.log(`(failed: ${detail}. Your message was not kept; try again.)`);
    }
  } finally {
    current = null;
  }
}

process.on("SIGINT", () => {
  if (current) current.abort(); // stop the reply, stay in the chat
  else process.exit(0); // nothing running: Ctrl+C leaves
});

console.log(`chatlab chat  (${defaultModel} at ${baseUrl})  type /help`);
const rl = readline.createInterface({ input: process.stdin });
const interactive = process.stdin.isTTY;

process.stdout.write("you> ");
for await (const raw of rl) {
  const line = raw.trim();
  if (!interactive) console.log(line); // piped input is not echoed by a terminal
  if (line) {
    if (line.startsWith("/")) {
      if (runCommand(line) === "quit") break;
    } else {
      await reply(line);
    }
  }
  process.stdout.write("you> ");
}
console.log(`\nbye. ${usageLine()}`);

The file has state at the top (conversation, token tally, and current, the AbortController of the reply in progress), then runCommand, the reply loop, the SIGINT handler, and the main loop.

Notice what this file does not contain: no fetch, no headers, no JSON parsing, no SSE. All of that lives in client.js and stream.js. That is the payoff of building the toolkit in layers. The chatbot is about 120 lines because the hard parts are somewhere else. Notice too that cli.js does not use a key: it only knows config.js. Hold that thought until the next chapter.

To use a real provider later, you change nothing in this file. Set the three environment variables from chapter 2 (LLM_BASE_URL, LLM_API_KEY, LLM_MODEL), keep the key out of your code, and read your provider's documentation, because the request shape can differ in small ways.

12. A Real Test for the Chatbot

Since the program reads from stdin and the mock is free, you can write automated tests that run the real chatbot against the mock. Node has a built-in test runner, node:test, so you need no packages. Create test/cli.test.js:

import test from "node:test";
import assert from "node:assert/strict";
import { spawn } from "node:child_process";
import { startMock } from "../mock-llm.js";

const server = await startMock(8821);
test.after(() => server.close());

// Run the CLI, feed it `input` on stdin, resolve with everything it printed.
function chatWith(input, env = {}) {
  return new Promise((resolve) => {
    const child = spawn("node", ["src/cli.js"], {
      env: { ...process.env, LLM_BASE_URL: "http://localhost:8821", ...env },
    });
    let out = "";
    child.stdout.on("data", (chunk) => (out += chunk));
    child.on("close", () => resolve(out));
    child.stdin.end(input);
  });
}

test("remembers the name within a session", async () => {
  const out = await chatWith("my name is Sok\nwhat is my name?\n");
  assert.match(out, /Your name is Sok\./);
});

test("/reset makes it forget", async () => {
  const out = await chatWith("my name is Sok\n/reset\nwhat is my name?\n");
  assert.match(out, /I do not know your name/);
});

test("a failed request does not stop the chat", async () => {
  const out = await chatWith("hi\n/usage\n", { LLM_API_KEY: "wrong" });
  assert.match(out, /API error 401/);
  assert.match(out, /0 requests/);
});

Each test starts a fresh chatbot process, feeds it a script on stdin, waits for it to exit, and checks the text it printed with a regular expression. The mock server runs inside the test file on its own port (8821 here, any free port will do), and the after hook closes it so the test run can finish.

The helper uses spawn and waits for close, not spawnSync: a synchronous wait would freeze this process, and the mock server lives in it, so nothing could answer.

Run it:

node --test --test-reporter=spec test/*.test.js
# ✔ remembers the name within a session (475.769083ms)
# ✔ /reset makes it forget (814.143958ms)
# ✔ a failed request does not stop the chat (68.692958ms)
# ℹ pass 3
# ℹ fail 0

The test names, the counts and the check marks are stable. The milliseconds vary on every run and on every computer. The first two take longer because the mock streams word by word, with a 40 ms pause between words.

Now you can change cli.js with confidence. If you break /reset, a test tells you in two seconds, with no typing.

13. Try It Yourself

1. Add /model <name>. Let the user switch models during a chat. You will need to pass model into chatStream. Test it with /model mock-error, see the failure message, then switch back to /model mock-1. Does the conversation survive?
2. Add /save <file> and /load <file>. Use the JSON save-and-load idea from chapter 3. After /save chat.json, quit, start again, /load chat.json, and ask "what is my name?". Which part of the state do you also need to save so a reload gives the same persona?
3. Make /system reset the chat. Change it so that setting a new system prompt also clears the history, and print a different message. Which behaviour do you prefer, and why? Think about what a half-finished conversation under an old persona does to a new one.
4. Show the cost of each turn. After every reply, print a dim line such as (in 31, out 8). Then add an /estimate command that uses an example price per million tokens (label it clearly as an example) to turn the tally into a rough money amount. Chapter 7 does this properly.
5. Fix the leading assistant message. After trim(), if the first turn is from the assistant, drop it. Write a test in test/cli.test.js using CHAT_MAX_TURNS=2 that checks the CLI still answers correctly after the history is trimmed.
6. Write a second test. Pipe in /system Talk like a pirate. and then a greeting, and assert that the reply starts with Arr!. Then add a test that sends a 401 and checks the program exits cleanly with the exit code 0.

14. Quick Review: Common Questions

Questions beginners often ask about this chapter. Try answering each one yourself first, then click the question to open the answer.

Where to go next. Your chatbot works, but only you can use it, and only on your own computer, because the API key is in your environment. The moment you want a web page that other people can use, you hit the hardest rule in this series: the key must never reach the browser. The next chapter, API Keys and a Backend Proxy, shows why, and builds a small server that holds the key for you.