Backend Fundamentals

3.2 HTTP - the envelope every request travels in

Every HTTP request has: a method, a URL, headers, and (often) a body.

   METHOD   URL
   POST     http://localhost:1234/v1/chat/completions
   ┌─────────────────────────────────────────────┐
   │ HEADERS                                      │
   │  Content-Type: application/json              │
   │  Authorization: Bearer sk-...                 │
   ├─────────────────────────────────────────────┤
   │ BODY (the actual data, usually JSON)          │
   │  {"model": "...", "messages": [...]}          │
   └─────────────────────────────────────────────┘

Think of an HTTP request like a physical letter: the URL is the address on the envelope, the method is a stamped instruction like "please reply" vs "please deliver this package," headers are metadata written on the outside of the envelope (what language it's in, who's allowed to open it), and the body is the letter itself, sealed inside.

The common methods you'll actually use:

A GET request typically has no body - everything it needs is in the URL itself (and maybe headers), because it's just asking to read something, not send new data to be processed. A POST almost always has a body, because you're handing the server a payload to act on.

Status codes tell you what happened:

A few specific codes worth recognizing on sight, because you will see them constantly while working with LLM APIs:

Common mistake: treating every non-200 response as a Python exception. requests does not raise an error automatically just because the server said "400 Bad Request" - it hands you back a normal Response object either way. You have to check response.status_code yourself, or explicitly call response.raise_for_status() if you want it to throw. Beginners often write code that silently "succeeds" while actually receiving an error payload in response.json().

3.3 JSON - the shared language everything speaks

JSON (JavaScript Object Notation) is just text, formatted so that both a Python dict and a JavaScript object (and Java, and Rust, and everything else) can read/write it identically. It's the universal contract we described in Chapter 0.

   Python dict                    JSON text (what actually travels               Python dict
   (in memory)                     over the network, as raw bytes)              (in memory, elsewhere)
   ┌────────────────┐             ┌──────────────────────────┐               ┌────────────────┐
   │ {"name":"Ada", │  json.dumps │ '{"name": "Ada",         │   json.loads  │ {"name":"Ada", │
   │  "age": 36}    │────────────▶│   "age": 36}'            │──────────────▶│  "age": 36}    │
   └────────────────┘             └──────────────────────────┘               └────────────────┘
      YOUR MACHINE                    THE WIRE / DISK                          THEIR MACHINE

The key insight: JSON is text. It doesn't know about Python dicts, or JavaScript objects, or Java HashMaps - it's a lowest-common-denominator format that every language independently knows how to translate into its own native data structure. That translation step has names in every language:

JSON's data types, and how they map to Python

JSON only has six kinds of values, and every one of them maps onto something you already know in Python:

Notice JSON has no concept of a Python tuple, set, or custom class - if you try to json.dumps() one of those directly, it'll either silently convert it (tuples become arrays) or raise a TypeError (sets and custom objects, unless you teach it how). This matters the moment you build your own agent's memory or tool-call structures in Chapter 6: stick to dicts, lists, strings, numbers, booleans, and None for anything that needs to cross a JSON boundary.

A slightly bigger example, showing nesting (objects inside arrays inside objects - exactly the shape you'll see in real LLM API responses):

This dumps/loads round trip is exactly what happens every time you send a prompt to an LLM and get a reply back - your requests library and the openai SDK do this JSON conversion for you automatically, but now you know what's happening underneath.

Common mistake: forgetting that response.json() can itself throw if the server didn't actually send valid JSON back (for instance, an HTML error page from a proxy, or an empty body). If you ever see json.decoder.JSONDecodeError, the fix is to first print(response.text) to see the raw text you actually got back, before assuming it was JSON at all.

3.4 Calling an API from Python

   INPUT (Python dict)         TRANSFORM                          OUTPUT (Python dict)
   {"question": "..."}    requests.post(url, json=...)      response.json()
                           ───────────────────────────▶      {"json": {"question": "..."}, ...}
                             (auto dict→JSON out,
                              auto JSON→dict back)

This four-line pattern - build a dict, POST it, call .json() on the response - is the entire skeleton of Chapter 5's first LM Studio call. Nothing new will actually be new; it'll be this, with a different URL and a different-shaped dict.

The two ways to send data with requests.post, and why they're different

requests.post(url, json=...) is convenient shorthand. It's worth seeing what it's shorthand for, because you'll eventually hit an API that wants form data instead of JSON:

data= sends raw bytes/text as the body with no assumptions about format; json= is a convenience wrapper that does the json.dumps plus header-setting for you. Every LLM HTTP API you'll touch in this book (LM Studio, OpenAI, Anthropic) expects JSON bodies, so you'll almost always use json=.

Sending headers, like an API key

Real APIs (unlike LM Studio's local, keyless server) require an Authorization header proving who you are:

Bearer here just means "the token that follows proves I'm authorized" - it's a convention, not magic. Chapter 5 shows you how to load that key from your .env file (from Chapter 2) instead of ever typing it directly into your source code.

3.5 Building a tiny backend yourself (so "server" stops being a black box)

You don't need to build LM Studio's server - it's already built. But building one tiny one yourself for five minutes will make everything else in this book click.

pip install fastapi uvicorn

uvicorn server:app --reload --port 8000

Two separate things are happening in that command worth naming: FastAPI is the framework that lets you describe endpoints (@app.post("/echo") and what function should run when that URL is hit). Uvicorn is the actual program that opens a network port, listens for incoming connections, and hands each request to FastAPI to figure out what to do with it. FastAPI without Uvicorn (or something like it) is just Python code sitting there - Uvicorn is what makes it reachable over a network at all.

Now, from another terminal (or your Python script):

You just ran a backend on your own machine, on localhost (meaning: "this same computer, not the internet"), and talked to it over HTTP. This is precisely what LM Studio does - except instead of @app.post("/echo") echoing your input, it has @app.post("/v1/chat/ completions") running an actual language model and returning its generated text. Same pattern, bigger box.

   YOUR TOY SERVER                          LM STUDIO'S SERVER (Chapter 5)
   POST /echo                               POST /v1/chat/completions
   {"hello": "world"}                       {"model": "...", "messages": [...]}
          │                                         │
          ▼                                         ▼
   returns input unchanged                  runs the LLM, returns generated text

Adding a second endpoint

To really see this is "just a program that responds to different URLs," add another route to server.py and restart:

/health endpoints like this one are extremely common in real backends - they exist purely so that monitoring tools (or you) can ask "are you alive?" without asking the server to do any real work. LM Studio, OpenAI, and most production APIs all expose something similar.

Common mistake: forgetting --reload while iterating on server.py, then editing the file and wondering why your changes don't take effect - Uvicorn is still running the old version of the code it loaded at startup. --reload tells it to watch the file and restart automatically; without it you must stop (Ctrl+C) and rerun the uvicorn command yourself after every edit.

3.6 Async - why LLM code always has async/await sprinkled in it

Calling a model can take several seconds. If your program just sits and blocks the whole time, it can't do anything else - no handling other users, no updating a progress bar, nothing.

   SYNCHRONOUS (blocking)                  ASYNCHRONOUS (non-blocking)

   call model ──(waiting... 3 sec)──▶ done   call model ──▶ (frees up thread to do other work)
   nothing else happens meanwhile                          ...3 sec later, result arrives, resumes

Picture a restaurant with one waiter. Synchronous service: the waiter takes your order, stands at the kitchen window doing nothing until your food is ready, then serves it, and only then goes to take the next table's order. Asynchronous service: the waiter takes your order, hands it to the kitchen, immediately goes to take other tables' orders while your food cooks, and comes back to deliver it the moment it's ready. Same waiter, same amount of total work, but the asynchronous waiter serves far more tables per hour because they never stand around idle.

async def marks a function as one that can pause while waiting (at every await) instead of blocking everything. await is the pause point itself - it means "give control back to whoever's coordinating tasks until this particular thing finishes." You cannot use await outside an async def function; and calling an async def function without await (or without asyncio.run) doesn't actually run it - it just gives you back an unexecuted "coroutine" object, which is one of the most common async bugs.

Why does this matter for a single script that only calls one model at a time?

It might not, if you truly only ever do one thing sequentially. Async pays off the moment you need to do more than one slow thing concurrently - for example, calling three different models at once and waiting for whichever answers first, or handling multiple users' requests to your own backend simultaneously (exactly what uvicorn from 3.5 does under the hood). Here's the concurrent case, which is where async actually earns its keep:

You'll see this pattern again in Chapter 7 whenever an agent asks two different backends for their answers to compare, or fans a request out across multiple tools at once.

You'll see this heavily when we do streaming in Chapter 5 - where the model's reply arrives word-by-word instead of all at once, and your code needs to process each piece as it lands rather than waiting for the whole thing.

yield (a generator) is Python's synchronous cousin of this idea - producing values one at a time instead of building the whole list first. LLM streaming APIs use this exact shape. Streaming and async solve two different problems that often show up together: streaming is about "give me the output piece by piece as it's produced" (the shape of the data), while async is about "let other work happen while I wait" (how time is spent). A real streaming LLM client usually combines both: an async def function that yields each token as it arrives, often written as async def stream_tokens(): ... yield token, called an async generator.

Common mistake: assuming async makes code faster by itself. It doesn't speed up any single slow operation - the network call to the model still takes exactly as long. What async buys you is the ability to do other things during that wait, not a shorter wait.

3.7 Recap - the pipe you'll use for the rest of the book

  your Python dict  →  json.dumps (automatic)  →  HTTP POST  →  server does work
                                                                       │
  your Python dict  ←  json.loads (automatic)  ←  HTTP response  ←────┘

Every single API call you make in this book - to LM Studio, to OpenAI, to Anthropic, to your own FastAPI server - is a variation on this exact picture: build a dict, decide the method and URL, send it as JSON over HTTP, get JSON back, parse it into a dict, read the field you need out of it. The dict's shape changes chapter to chapter (Chapter 4 covers exactly what shape an LLM expects and returns), but the pipe itself never does.

Chapter 4 gives you the vocabulary for what's actually inside that dict when the server is an LLM. Chapter 5 puts it all together with a real, running local model.

3.8 Check yourself

1. In the phrase "client/server," is the same machine always the client or always the server? Give an example from this chapter where the same laptop plays both roles at different moments.
2. What's the difference between the host, the port, and the path in a URL like http://localhost:8000/echo?
3. Your code calls requests.post(...) and the server returns status code 401. Does requests raise an exception on its own? What line of code would you add to make it raise one?
4. List the six JSON value types and their Python equivalents. Which common Python type has no direct JSON equivalent, and what usually happens if you try to json.dumps() it?
5. What's the actual job of Uvicorn versus the actual job of FastAPI, in the toy /echo server from 3.5?
6. Explain, using the restaurant-waiter analogy or your own, why async/await doesn't make a single slow network call finish any faster - and describe a scenario where it does reduce total wall-clock time.
7. What is the practical difference between a generator's yield and an LLM API's streaming response? Why do both show up together in real streaming client code?