← Back
Home › Systems › Agentic AI

mcp-host

Connects a language model to many tool servers at once, deciding which servers to trust, which tools the model sees and which calls need approval.

Claude Desktop

Open the mcp-host pagemcp-host Agent loopagentic-ai: Agent sandboxingagentic-ai: Context window managementagentic-ai: Keep an agent's growing history inside the model's window by truncating, clearing and summarizing old turns in a cheap-to-lossy ladder, at turn boundaries.›Model context protocolagentic-ai: JSON-RPC protocol exposing tools, resources and prompts from servers to LLM clients, with capability negotiationPrompt injection defenseagentic-ai: Treat every untrusted string as data, track where values came from, and make tool calls and rendered output depend on provenance, not on what the model says.›Tool callingagentic-ai: The model emits a structured call (id, name, JSON arguments) instead of prose; your code validates, runs it and returns a result tagged with the same id.›Tool retrieval (selecting toolsfor a request)agentic-ai: Index tool descriptions, filter by the caller's permissions, rank against the request and put only the top few definitions in context, in a stable order.›Hdr histogramdata-structures: Log-linear bucketed histograms with bounded relative error for latency percentilesDeterministic simulationtestingdistributed: Run the whole system in one thread under a seeded scheduler with simulated network, disk and clock, so every failure replays from its seed.›Primary-backup replicationdistributed: One node orders all writes and ships them to standbys; the acknowledgement rule (none, one, all) sets what a crash loses, and an external arbiter decides who takes over.›Process supervisionhardware-os: A parent process starts services, reaps their exits and restarts them under a delay and burst-limit policy, then stops them with SIGTERM before SIGKILL.›Embedding inferencemachine-learning: Serve a pretrained embedding model: tokenize, batch requests, run the encoder, pool and normalize vectors, cache them; no trainingJson rpcnetworking: JSON-RPC 2.0 requests, notifications and batchingServer-sent eventsnetworking: One long-lived HTTP response carries text events one way; each event may carry an id, so a reconnecting client names its last one and the server resumes.›Capability-based security(object capabilities)security: Authority is a handle you hold, not an identity you have: code can only act on objects it was explicitly given handles to, and handles can only be narrowed and passed on.›Coverage-guided fuzzingsecurity: Mutate inputs, run an instrumented target, and keep any input that reaches a new code edge; feedback turns blind guessing into incremental search.›Hardware backed keystoresecurity: Non-exportable keys in Secure Enclave, StrongBox, TPM: generation, use policies, biometric gatingJSON Schema validationsecurity: Compile a JSON Schema once, then check every untrusted document against it, within hard size and depth limits, and report failures as JSON Pointers.›Least privilege and isolationsecurity: Assume every component will be compromised and limit what it can then reach: narrow identities, separate accounts, sandboxed processes.›Mutual TLSsecurity: TLS where the client also proves a certificate key, so every connection carries a verified, rotatable machine identity.›OAuth 2 and OpenID Connectsecurity: Delegated authorization (OAuth 2) and login on top of it (OIDC); use authorization code + PKCE for every user-facing client.›Process sandboxingsecurity: Run risky work in a separate, deliberately powerless process and let a small privileged broker do the few things it needs, over a narrow pipe.›Secrets managementsecurity: Keep keys and passwords out of code and logs, give workloads short-lived identity instead of static keys, and make rotation routine.›Point-in-time recoverystorage: Periodic base backups plus a continuously archived change log let you rebuild the state as of any instant, then branch from there.›Exponential backoff with jittersystems: Retry after delays that grow geometrically and are randomized, so failed clients spread out instead of hammering a recovering service in synchronized waves.›Graceful drainingsystems: On a termination signal, stop taking new work, finish in-flight work until a deadline, tell clients to reconnect elsewhere, then exit; clients resume from a token.›Token samplingml-training: Turn a vector of scores into one chosen item with temperature, top-k, top-p, min-p, repetition penalties and a seed.›Egress proxy filteringsecurity: Forcing sandbox network traffic through a proxy with domain allowlists, TLS SNI checks and request loggingMicrovm isolationsecurity: Minimal KVM-based VMs (Firecracker-style) with tiny device model, fast boot and snapshot restorePrompt cachingagentic-ai: Reuse the attention keys/values of a repeated prompt prefix, so an agent's growing context is prefilled once and later turns pay only for the new tokens.›Byte-pair encoding (BPE)tokenizationmachine-learning: Learn a subword vocabulary by repeatedly merging the most frequent adjacent pair; byte-level BPE encodes any input losslessly with 32k-200k tokens.›Text chunkingmachine-learning: Cut documents into token-bounded pieces along structure boundaries, keeping offsets and section paths, so each piece embeds well and can be cited.›Text classificationmachine-learning: Assign one or more labels to a text; climb a ladder from regexes to linear models on n-grams, embeddings, fine-tuned encoders and LLM prompts, only as far as the data and budget demand.›Html sanitizationsecurity: Allowlist-based HTML sanitizing of untrusted markupTaint trackingsecurity: Track untrusted sources to sensitive sinks through dataflowUnicode confusablessecurity: UTS #39 maps every string to a "skeleton" so look-alikes collide, and restricts script mixing so most spoofs never get registered.›Constrained decodingagentic-ai: Force LLM output to match a regex, JSON schema or grammar by masking, at every step, the tokens that would leave the language.›Embeddingsmachine-learning: A learned table mapping discrete ids to dense vectors: gather forward, sparse scatter-add backward, row-wise optimizers to keep huge tables cheap.›+ 26 moreDNS message format and clientlookupsnetworking: Binary query/response format of DNS (RFC 1035) and how a stub client uses it: record types, name compression, UDP/TCP/EDNS sizes, TTL caching.›Http connect tunnelingnetworking: HTTP CONNECT tunnels for proxying TLS and arbitrary TCPPublic Suffix Listnetworking: Find the registrable domain (eTLD+1) of a host name by matching the longest rule in a community-maintained list of shared suffixes.›Glob matchingsearch-sort: Glob/gitignore pattern matching without backtracking blowupHardware virtualization kvmhardware-os: VT-x/AMD-V via KVM: vCPU run loop, VM exits, MMIO emulationNested paging epthardware-os: EPT/NPT two-level address translation for guestsVirtio deviceshardware-os: Virtqueue rings for paravirtual block, net and vsock devicesRadix triedata-structures: Compressed prefix tree for path routing, domain matching and prefix lookupsPaged KV cacheml-training: Store each sequence's attention keys/values in fixed-size blocks mapped through a block table, like virtual memory pages; near-zero fragmentation and copy-on-write sharing.›LRU and CLOCKsystems: LRU evicts the least recently used entry via hash map plus linked list; CLOCK approximates it with a reference bit, so hits take no lock.›Calibrationmachine-learning: Make predicted probabilities match observed frequencies: measure with ECE, fix post hoc with temperature, Platt or isotonic scaling on held-out data.›Linear and logistic regressionmachine-learning: Fit a weighted sum of features to a number (least squares) or a probability (logistic); the baseline every model must beat.›+ 22 moreReference countingcompilers: Keep a count of owners in each object, free it when the count hits zero; deterministic and local, but pays on every copy and leaks cycles.›Loss functionsmachine-learning: The scalar a model minimizes: cross-entropy for classes and tokens, MSE or Huber for values, each with a stable form and simple gradient.›L-BFGSnumerics: Quasi-Newton minimizer that approximates the inverse Hessian from the last m gradient differences via the two-loop recursion; O(m·n) memory and work per step.›+ 11 more+ 2 more

How it is built

Architecture overview

An MCP host is the AI application that connects a language model to many tool servers at once. It reads a list of servers from configuration, starts or dials each one, and keeps one client object per server, each with its own connection and its own negotiated capabilities (source: MCP architecture overview). It merges what the servers offer into one catalogue, shows the model a selection of it, routes each tool call the model makes back to the right server, and puts the result into the next prompt. Everything a server sends is outside the host's trust boundary: tool descriptions, results and prompts are text that will be read by a model that holds other tools.

The one idea that makes it manageable: the host owns the policy, the servers own the work. The protocol only moves messages. Which servers to trust, which tools the model sees, which calls need a human, how large a result may be, when to reconnect: all of that is host code, and each of those decisions is a safety or cost decision.

Parts:

 config (user / project / org)         ┌──────────────── host process ─────────────────┐
 servers: A (stdio) B (https) C (https)│ connection manager ── client A ── spawn ──► server A (child)
                                       │   │ reconnect, backoff ─ client B ── HTTPS ──► server B (remote)
                                       │   ▼                     client C ── HTTPS ──► server C (remote)
                                       │ tool registry  (A.x, B.y, C.z)  ◄── tools/list per client
                                       │   │ select + schema                 
 user ◄── confirm ───────────────────► │ model loop ──► LLM API ──► tool_use ─► route ─► tools/call
                                       │   ▲ result: size cap, provenance tag, append to context
                                       └───────────────────────────────────────────────┘

How the parts fit together

Connection manager

The host holds a list of server definitions. In Claude Desktop that is a JSON file (claude_desktop_config.json) with an mcpServers map from a name to a command and args (source: MCP docs, Connect to local MCP servers). In Claude Code the same idea has three scopes: local (private to you in one project), project (a .mcp.json in the repository, shared by version control) and user (all your projects) (source: Claude Code docs, MCP). The scopes encode trust: a project file arrives from a cloned repository, so Claude Code does not connect a project server until the user has approved it, and an untrusted folder cannot approve its own servers.

Each definition gets one client. A stdio server is a child process serving one client; a remote server serves many. The lifecycle is the protocol's: under revision 2025-11-25 a client sends initialize with its version and capabilities, the server answers with its own, then notifications/initialized; under 2026-07-28 the handshake is gone and each request carries the version and capabilities, with server/discover available to learn them up front (source: MCP spec lifecycle and changelog). A host that talks to servers of both eras detects the era and falls back (the 2026-07-28 spec has a compatibility matrix).

Connections fail, and the manager is where that is handled. Exponential backoff with jitter is the rule: Claude Code retries a first HTTP connection up to three times on transient errors (5xx, refused, timeout) but not on authentication or not-found errors, which need a configuration change; it reconnects a dropped remote server up to five times starting at one second and doubling each time, then marks it failed and tells the user; stdio servers are local processes and are not reconnected automatically (source: Claude Code docs, MCP). Jitter matters for a hosted host: thousands of sessions reconnecting in step after a restart is a self-inflicted outage.

For a stdio server the manager is also a parent process, so Process supervision applies: reap exits, bound restarts, send SIGTERM then SIGKILL at the end. The spec's shutdown order for stdio is exactly that: close the child's stdin, wait, SIGTERM, then SIGKILL. Graceful draining is the host-side version: before exiting, let in-flight calls finish or cancel them, so a half-done write is not left behind.

Transport clients

The two standard transports need two kinds of client. The stdio client writes newline-delimited JSON to the child's stdin and reads the child's stdout; it must treat anything on stderr as logging, not as an error (source: spec, Transports). The Streamable HTTP client sends every message as a POST, with an Accept header naming both application/json and text/event-stream, and must handle either reply: one JSON object, or an event stream (Server-sent events) carrying progress and ending with the response. Under 2025-11-25 it may also open a GET stream and resume with Last-Event-ID; under 2026-07-28 both are gone, and a stream that breaks loses the request, which the client re-issues as a new one. The client times out every request and cancels when it gives up.

Tool registry

Each client reports its tools with tools/list (paginated with a cursor). The registry merges them into one catalogue and has to solve four problems the protocol leaves open:

  • Names. Two servers may both offer search. The registry namespaces by server (Claude Code uses mcp__<server>__<tool>) and enforces the model API's name rules.
  • Validation. The registry keeps each tool's inputSchema, and checks the model's arguments against it (JSON Schema validation) before sending the call. A malformed call is answered locally with an error the model can fix, keeping bad input away from the server.
  • Selection. Every tool definition costs tokens on every request and a model chooses worse as the menu grows. Tool retrieval is the remedy: keep descriptions in an index, rank them against the request, expose the top few. Claude Code's tool search does this by loading only tool names and server instructions at session start and fetching definitions on demand; it cuts descriptions at 4,096 characters for upfront tools and 16,384 for deferred ones, so a server author should put the words a model would search for first (source: Claude Code docs, MCP tool search). Word ranking is cheap; Embeddings help when the user's wording differs from the description.
  • Change. A server that declared listChanged notifies when its tools change. Editing the exposed list mid-conversation breaks the prompt cache, so apply changes at turn boundaries and keep the order stable.

Model loop and context

The loop is Tool calling around the model: send the conversation and exposed tools, receive text or tool_use blocks, run the calls, return one tool_result per call, repeat (the agent-loop (no page yet), no page yet). MCP changes one thing here: results are produced by programs the host does not control, so their size and shape are unbounded. Context window management therefore starts at the registry boundary. Claude Code warns above 10,000 tokens per result, caps output at 25,000 by default (MAX_MCP_OUTPUT_TOKENS), saves text over 50,000 characters to a file, and moves a call still running after two minutes to a background task (source: Claude Code docs, MCP).

The spec adds interaction rules a host must implement. Hosts SHOULD show which tools are exposed to the model, show when one is invoked, and let a human deny calls. Elicitation lets a server ask the user for input as a form the host renders from a schema, or as a URL the user opens in a browser so secrets never pass through the host. Roots and sampling (a server asking the host's model for a completion) are deprecated as of 2026-07-28 and new hosts should not add them (source: MCP changelog).

Credentials

Remote servers use OAuth 2 and OpenID Connect as the MCP authorization spec defines it. The host is the OAuth client: it finds the authorization server from the server's protected-resource metadata (RFC 9728, usually announced in the WWW-Authenticate header of a 401), uses the authorization-code flow with PKCE, sends the resource parameter (RFC 8707) so the token is bound to that one server, and must never send a server a token that was not issued by that server's authorization server. A request that fails with insufficient scope triggers a step-up: request more scope and retry. Under 2026-07-28 the host must also key stored client credentials by the authorization server's issuer, not reuse them with another, and Client ID Metadata Documents are preferred over dynamic client registration (source: MCP spec, Authorization).

Where tokens live is the host's job. A refresh token in a plain config file is a long-lived credential; keep it in the operating system's store, ideally hardware-backed (Hardware-backed keystore, no page yet), and follow Secrets management for the server definitions themselves: environment variable expansion in a shared .mcp.json lets the file name a variable instead of containing the key (Claude Code supports this). A stdio server gets its credentials from the environment the host sets for it, not from OAuth.

Trust and confinement

A local stdio server is a program started with the user's rights: installing one is running code. Remote servers are not code on the machine but they supply text to the model. The host's defences therefore split in two.

Against what the servers say. Tool descriptions and results are untrusted input. A hostile server can put instructions in a description or result, or change a description after the user approved it; a result from one server can steer the model into calling a tool from another. Prompt injection defense is the answer, and it is structural: tag every value with the server it came from, do not let text from one server decide the arguments of a sensitive call to another without a check, and keep a human confirmation on actions with side effects. The spec says tool annotations must be treated as untrusted unless the server is trusted. Risk peaks when one session holds private data, untrusted content and a way to send data out; limiting that combination beats filtering text.

Against what a server can do. Run local servers under Process sandboxing with only the folders and network they need (agent-sandboxing (no page yet), no page yet, is the agent side) and Least privilege and isolation. Hand a server narrow handles, not ambient authority (Capability-based security): the filesystem server in the docs receives the directories to expose as arguments, which is this pattern. For hops between hosted components, Mutual TLS gives each side a verified identity.

Hosted-host operations

A desktop host keeps state in files. A host run as a service (a web assistant with connectors) also keeps per-user connections, tokens and conversations, so the generic blocks apply: Point-in-time recovery and Primary-backup replication for that store; an HDR histogram (no page yet) of tool-call latency per server so one slow server is visible in the tail; Deterministic simulation testing of the client state machine (connect, drop, reconnect, cancel, re-authorise) under injected faults; and Coverage-guided fuzzing of the message parsers, because every byte from a server is untrusted.

Data and control flow

Startup and one tool call:

  1. Read configuration from all scopes; ask approval for project servers; resolve duplicates by precedence.
  2. In parallel, spawn each stdio server and dial each remote one; one failure must not block the others.
  3. For each connected client, run the handshake (2025-11-25) or server/discover (2026-07-28), then tools/list, following cursors; read the server's instructions text.
  4. Build the registry with namespaced names. With tool search on, the model sees names and instructions only; otherwise full definitions.
  5. The user writes a request; the loop sends conversation plus the selected tools to the model API. Time: dominated by the model.
  6. The model emits tool_use for server_b.search. The registry validates arguments against the schema; the policy layer checks the allow list and, if needed, asks the user.
  7. The client for server B sends tools/call (a POST with a bearer token). A 401 starts the OAuth flow; a timeout sends a cancellation and returns an error result.
  8. The result is capped in size, tagged with its origin, appended as a tool_result; the loop calls the model again.

Server needs the user (2026-07-28): the server returns an InputRequiredResult with an elicitation request; the host renders a form or opens a URL and retries the same call with the answer attached (source: MCP changelog).

Scaling

In the order the limits arrive:

  1. Context cost of tools. Add servers and the tool list eats the window and degrades choice first. Lever: tool retrieval or deferred loading, stable ordering for the cache. This bites long before any CPU limit.
  2. Result size. Large results bury the context. Lever: caps, write-to-file, ask servers for paging, clear old results (Context window management).
  3. Latency of one slow server. The model waits for the slowest call in a turn. Lever: per-server timeouts, background tasks for long calls, parallel calls to different servers.
  4. Connection count in a hosted host: one client per server per session means many idle connections. Lever: pool per user (stateless HTTP in 2026-07-28 helps), connect lazily on first use.
  5. Refresh storms after an outage. Lever: jittered backoff, one refresh in flight per credential.

Failure modes

FailureSymptomCauseFix
Server fails silentlythe model does not know a tool exists and answers without itconnection failed; host never told the modelsurface failed servers to the model and the user; /mcp-style status
Reconnect storma restarted server is hit by all clients at oncefixed retry intervalexponential backoff with jitter, cap on attempts
Tool-name clashwrong server called, or request rejectedtwo servers share a tool name, name too longnamespace by server; validate names
Context floodbudget spent in one call, answers degradeunbounded tool resultsize cap, file offload, summarisation
Tool poisoningmodel leaks data or calls an unrelated toolinstructions hidden in a tool description or resultprovenance tags, confirm side effects, review before approving a server
Description drifta tool behaves differently after approvalserver changed its tool list or descriptionre-prompt on change, pin or hash definitions
Token sent to the wrong serverdata exposed to a third partytoken reused outside its audiencebind with the resource parameter; key credentials by server and issuer
Cache invalidationcost jumps after adding a server mid-chatexposed tool list changed between turnsapply tool changes at turn boundaries, keep order stable
Orphan child processstdio server still running after the host exitshost died without the shutdown sequenceclose stdin, SIGTERM, SIGKILL; supervise children

Real implementations

namewhat it isrepolicencenotes
Claude Desktopdesktop assistant, local servers from a JSON config filenone public; setup guideproprietaryexample host in the MCP local-server guide
Claude Codeterminal coding agent; three config scopes, OAuth, tool search, output limitsanthropics/claude-codeproprietary (all rights reserved)docs: MCP
Visual Studio Codeeditor with MCP support in its chat featuresmicrosoft/vscodeMITnamed as an example host in the MCP architecture docs
Gooseopen-source local agent with extensions over MCPblock/gooseApache-2.0README: extensions via MCP
Clinecoding agent in the editor with MCP serverscline/clineApache-2.0
LibreChatself-hosted chat UI with MCP servers and OAuth sessionsdanny-avila/LibreChatMITmulti-user, hosted-style
OpenAI Agents SDKagent library that can attach MCP serversopenai/openai-agents-pythonMITMCP docs

The alternative: bake the few tools you need into the application as plain Tool calling functions. That removes third-party trust and suits a small, fixed tool set. Choose MCP when users bring their own tools or many applications share them.

Sources

Similar systems

Share the most building blocks with this one.

agent-runtime 56 %mcp-server 48 %agent-gateway 40 %computer-use-agent 37 %reverse-etl 35 %