Multi-Provider Architecture

7.1 The problem, concretely

   LM Studio (OpenAI-style)                 Anthropic (its own style)
   client.chat.completions.create(          client.messages.create(
       model=...,                               model=...,
       messages=[                               max_tokens=1024,
         {"role":"system","content":...},       system="...",              ← system is SEPARATE, not in the list
         {"role":"user","content":...},         messages=[
       ]                                           {"role":"user","content":...},
   )                                             ]
                                             )

Same intent ("send a conversation, get a reply"), different shape. If run_agent calls these directly, every provider you add means editing your core agent logic. That's fragile - exactly the problem Chapter 1.5's inheritance example was foreshadowing.

Imagine you didn't build the abstraction, and instead wrote your agent loop directly against the OpenAI-shaped call. The first time you wanted to add Anthropic support, you'd probably reach for something like this:

This works for two providers. Now imagine adding a third, a fourth, a fifth - every one of them requires editing this same function, and the function grows a new elif branch every time, each one mixing "how do I run the loop" with "how does provider X's API happen to be shaped." That's the exact anti-pattern Chapter 1.5 warned about with deeply nested if/elif chains standing in for what should be separate objects. The fix, as always, is to separate "the thing that varies" (how each provider talks) from "the thing that doesn't" (the loop's logic).

   BEFORE (one giant function, growing forever):        AFTER (one small function per provider):
   def run_agent(provider_name, messages):              class LMStudioProvider: .chat(...)
       if provider_name == "lmstudio": ...               class OpenAIProvider: .chat(...)
       elif provider_name == "openai": ...                class AnthropicProvider: .chat(...)
       elif provider_name == "anthropic": ...             class MistralProvider: .chat(...)   <- just add one more
       elif provider_name == "mistral": ...               run_agent(provider, messages) never changes
       ...

7.2 The fix: one shared contract, many adapters

                     ┌─────────────────────────────┐
                     │   BaseProvider (the contract) │
                     │   .chat(messages) -> str       │
                     └─────────────────────────────┘
                        ▲            ▲            ▲
                        │            │            │
             ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
             │ LMStudio      │ │ OpenAI        │ │ Anthropic     │
             │ Provider      │ │ Provider      │ │ Provider      │
             │ (talks OpenAI-│ │ (talks OpenAI │ │ (translates to│
             │  style API to │ │  cloud API)   │ │  Anthropic's  │
             │  localhost)   │ │               │ │  own shape)   │
             └──────────────┘ └──────────────┘ └──────────────┘

Your run_agent function only ever talks to BaseProvider. It has no idea, and doesn't need to care, which one is actually plugged in underneath - precisely the inheritance pattern from Chapter 1.5, now doing real work.

This pattern has a name in software design: it's called the Adapter pattern. An adapter's entire job is to sit between "what my code wants to call" and "what the real thing actually looks like," translating one into the other. A physical power adapter is the same idea: your laptop charger wants a certain shape of plug; the wall socket offers a different shape; the adapter in between doesn't change electricity at all, it just changes the shape of the connection so the two sides can talk.

ABC and @abstractmethod make this a formal contract: any subclass that forgets to implement chat will error immediately, rather than failing mysteriously later.

To see this concretely, try defining a broken subclass that forgets to implement chat:

Python refuses to even construct BrokenProvider() - it raises a TypeError at the moment you try to instantiate it, listing exactly which abstract method is missing. Compare that to what would happen without ABC: you'd only discover the missing chat method the first time run_agent tried to call it, possibly deep into a live session, possibly in production. ABC turns a "fails at the worst possible time" bug into a "fails immediately and obviously" bug - this is generally a trade worth making whenever you're designing a contract multiple classes must satisfy.

Common mistake: thinking ABC prevents incorrect implementations. It doesn't - it only guarantees the method exists. Python has no way to check that your AnthropicProvider.chat actually returns something with the same shape as LMStudioProvider.chat. That discipline is on you; it's why the docstring on BaseProvider.chat describing exactly what it should return matters as much as the method signature itself.

7.3 Adapter 1 - LM Studio (you already wrote this in Chapter 5/6, now wrapped)

Nothing here is new - this is the exact client.chat.completions.create(...) call from Chapter 6.4, just moved inside a method with a name (chat) that matches the shared contract. The whole trick of the Adapter pattern is often this anticlimactic: for the "easy" cases, the adapter is barely more than a thin wrapper renaming things to match the contract. The value shows up when a provider's API genuinely differs, which is exactly what Section 7.5 covers.

7.4 Adapter 2 - OpenAI cloud (near-identical shape, different base_url and a real key)

Notice LMStudioProvider and OpenAIProvider are almost identical - because LM Studio was designed to mimic OpenAI's shape (Chapter 5.3). The real translation work happens next.

It's worth noticing why nearly identical code still deserves two separate classes instead of one class with an if is_local: ... branch inside it. Each class represents one clear responsibility ("talk to this specific kind of server, with this specific kind of authentication"), and if OpenAI's cloud API ever adds a feature LM Studio's local server doesn't support (say, image inputs), you only touch OpenAIProvider, leaving LMStudioProvider untouched and guaranteed not to break. Duplication that is this small is often a fine price to pay for that isolation - a lesson worth remembering before reflexively "DRY-ing up" every few lines of similar code.

7.5 Adapter 3 - Anthropic (here the shapes genuinely differ, so the adapter earns its keep)

   INPUT (your uniform list)                    ADAPTER TRANSLATES                OUTPUT (provider-native call)
   [{"role":"system","content":"be terse"},      split system out,           system="be terse"
    {"role":"user","content":"hi"}]         ──▶  keep rest as messages   ──▶  messages=[{"role":"user","content":"hi"}]

This is precisely why the abstraction exists: your agent's core logic (Chapter 6) never has to know that Anthropic's contract is shaped differently - the adapter absorbs that difference.

Let's trace through the translation loop by hand so the "why" is completely concrete. Suppose messages coming from your Memory object looks like this:

Run this and you'll see the single system entry gets pulled out into its own variable, while everything else stays in order inside chat_messages - exactly matching what anthropic.Anthropic().messages.create(...) expects. This is the entire "why" of an adapter in one runnable example: your code upstream never needs to know this reshaping happened.

Common mistake: assuming every provider's reply shape matches too. Notice LMStudioProvider.chat returns response.choices[0].message (an object with a .content attribute and possibly .tool_calls), while AnthropicProvider.chat returns response.content[0].text (a plain string). If your run_agent loop assumes every provider returns the exact same object shape, it will break the moment you swap providers - Section 7.7 shows the small isinstance check needed to handle this safely, and the note at the end of this chapter explains why tool-calling makes this gap even bigger.

7.6 Config-driven provider selection - the pyproject.toml idea, applied to YOUR project

Just like pyproject.toml declares what your project needs without hardcoding it into every file, your agent should declare which provider to use in one config place, not scattered through code.

pip install python-dotenv

# .env  (never commit this file - see Chapter 2.5's .gitignore)
PROVIDER=lmstudio
LMSTUDIO_MODEL=your-model-id-here
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...

   change ONE line in .env:  PROVIDER=lmstudio  ─▶  PROVIDER=openai
                                   │
                                   ▼
   entire agent now runs against a different backend, zero code changes

This is the exact same "why" as package.json's dependencies or pyproject.toml's dependencies list from Chapter 2: declare your choice in one config place, let the tooling resolve it, instead of hardcoding it deep inside logic where it's easy to forget and hard to change safely.

Notice build_provider still has an if/elif chain - and that's fine here, because this function's entire job is exactly "given a name, produce the matching object." This is called a factory function: a small, centralized place where the mapping from "identifier" to "concrete object" lives, so that mapping only ever needs to be written once. The difference between this and the anti-pattern from Section 7.1 is where the branching lives: here it's isolated to one tiny function whose sole purpose is selection, not smeared throughout your agent's actual logic.

   FACTORY FUNCTION PATTERN:

   "lmstudio"   ──▶  build_provider() ──▶  LMStudioProvider(...)
   "openai"     ──▶  build_provider() ──▶  OpenAIProvider(...)
   "anthropic"  ──▶  build_provider() ──▶  AnthropicProvider(...)

   the CALLER of build_provider() never needs to know these class names exist

You can test the factory in isolation, without any real API keys, by monkeypatching the environment:

Common mistake: committing a .env file with real API keys to version control. Chapter 2.5 already covered .gitignore for exactly this reason - an API key leaked in a public GitHub history can rack up thousands of dollars of usage on your account within hours of a bot scraping it. Always double-check .env is listed in .gitignore before your first commit, not after.

7.7 Rewiring the Chapter 6 agent loop to use the abstraction

Swap .env's PROVIDER value, run again, and the exact same run_agent and Memory code now talks to a completely different company's servers. That's the payoff of the whole chapter.

Look closely at getattr(message, "tool_calls", None) and content = message if isinstance(message, str) else message.content. These two lines are doing quiet, important work: they're the small seams where the loop tolerates the fact that LMStudioProvider.chat returns an object with attributes, while AnthropicProvider.chat (as written so far) returns a plain string. getattr(obj, "name", default) is a defensive way to read an attribute that might not exist - instead of crashing with AttributeError when a plain string has no .tool_calls, it just falls back to None, which the if correctly treats as "no tool calls requested."

   getattr(message, "tool_calls", None) applied to two different objects:

   message = <ChatCompletionMessage with .tool_calls=[...]>  ──▶  returns [...]
   message = "4."   (a plain Python string)                  ──▶  returns None  (strings have no .tool_calls)

The Production Standard: A Unified ProviderResponse

Rather than relying on getattr or isinstance checks inside your agent loop, production systems define a standardized response dataclass. This ensures that every provider returns the exact same data contract:

With this dataclass, every provider adapter takes responsibility for normalizing its raw API output before returning:

• AnthropicProvider maps block.type == "tool_use" to ToolCall(id=block.id, name=block.name, arguments=block.input).
• OpenAIProvider / GroqProvider / LMStudioProvider map message.tool_calls to ToolCall.
• Both compute latency_ms and extract prompt_tokens / completion_tokens.

Handling Streaming Tool Call Deltas

When streaming tokens (stream=True), providers return tool arguments in fragments across multiple chunks (e.g., {"exp, ression":, "2+2"}). The adapter accumulates these string fragments by tool index:

This ensures your agent loop stays 100% clean, never dealing with partial JSON parsing or vendor-specific response quirks.

Chapter 8 assembles everything from Chapters 1–7 into one real, runnable project folder.

Check yourself

• Explain, without looking back at the code, why a single run_agent function that branches on if provider_name == "openai": ... elif ... is harder to maintain than the BaseProvider approach, as the number of supported providers grows.
• What specific error does Python raise if you try to instantiate a subclass of BaseProvider that never defines chat? When does that error happen - at class definition time, or at instantiation time?
• Why does AnthropicProvider.chat need to separate the system message from the rest of the list before calling self.client.messages.create(...)? What would happen if you just passed the whole list, system message included, as messages=?
• build_provider() still contains an if/elif chain. Why is that acceptable here when it was called an anti-pattern in Section 7.1's "before" example? What's structurally different?
• Trace through getattr(message, "tool_calls", None) for both a plain string and an object with a .tool_calls attribute. What does each call return, and why doesn't the string case raise an exception?
• Sketch (in words or pseudocode) what a corrected AnthropicProvider.chat would need to do to return an object compatible with the tool_calls handling in run_agent.