Every major AI product you interact with runs on top of a layer of engineering that does not get named in the press release. That layer is the harness. The harness is where the intelligence actually lives.
A language model is a stateless text transformer. It has no memory of your last message. It cannot call APIs. It does not decide which model to use or when to stop. The harness does all of that. Think of the harness as the operating system and the model as the CPU.
Everything the model sees at inference time is assembled before the API call. This includes the system prompt (the model's standing instructions), retrieved documents from a vector or hybrid search index (RAG), conversation history, injected tool results, and structured memory.
The model cannot distinguish between "background knowledge" and "what I just looked up." From its perspective, it is all text in a context window. The harness decides what goes in and in what order.
RAG / system prompt / memoryModern model APIs support function calling: the model can emit a structured JSON payload describing a tool it wants to invoke. The harness catches that output, executes the function (a database query, an API call, a code runner), and feeds the result back into the next context.
The model does not "run code." It describes code it wants run. The harness runs it. This distinction matters enormously for security design.
function calling / code executionA base language model is completely stateless. It has no memory between API calls. Every conversation starts fresh unless the harness explicitly reconstructs context.
The harness typically manages three memory tiers: short-term (in-window conversation history), long-term (external storage, written and retrieved by the harness), and episodic (structured records of past sessions surfaced selectively at retrieval time).
short-term / long-term / episodicProduction harnesses rarely call a single model. A routing layer selects the right model for each request based on task type, privacy requirements, cost constraints, and latency targets. Fast, cheap models handle classification and summarization. Slower, more capable models handle complex reasoning.
Fallback chains handle provider outages. Gateway layers like Portkey and LiteLLM provide unified interfaces across providers.
Portkey / LiteLLM / model selectionThe harness is the primary control surface for AI safety in production. Intent classification gates which actions an agent is permitted to take. PII detection and tokenization prevent sensitive data from leaking into model context or logs.
Dual-Intent Runtime Authorization (DIRA) extends this by requiring that an agent's declared intent at request time matches its actual behavior at execution time, creating a cryptographic accountability layer for agentic workloads.
DIRA / intent classification / PIIProduction AI systems require the same observability discipline as any distributed service, with additional requirements unique to nondeterministic systems. Every model call should be traced: prompt, response, latency, cost, token count, and tool invocations.
Behavioral Drift Detection (BDD) extends classical observability by identifying when a model's response patterns shift over time, whether from model updates, prompt changes, or data distribution shift, before those shifts cause silent failures in downstream systems.
Langfuse / trace logging / BDDThe term appears constantly in AI coverage without being defined. That causes real confusion when selecting models, evaluating benchmarks, or designing routing logic. Here is a precise definition.
A frontier model is a model at the current capability edge across a broad task distribution. As of mid-2026, that tier includes GPT-4o and o3 (OpenAI), Claude Opus 4 and Sonnet 4 (Anthropic), Gemini 2.5 Pro (Google), and Llama 3.3 70B (Meta) on the open-weights side. What these models have in common: they are the first to hit human-competitive or superhuman performance on tasks that previous generations failed at consistently.
The frontier moves. GPT-3.5, which was genuinely impressive in late 2022, now gets beaten by models a fraction of its cost. Llama 2 has been superseded twice. "Frontier" is a relative position on a curve, not a grade stamped on a product. Any model described as frontier-class needs a date attached or the label is meaningless.
Closed-weights and open-weights both reach frontier quality. The distinction is not capability; it is control. Closed-weights models (Claude, GPT-4o, Gemini) are API-only. The provider runs the model, pushes updates, and manages safety filtering. You never touch the weights. Open-weights models (Llama 3.3, Mistral Large, DeepSeek R1) publish their parameters. You run them inside your own infrastructure, fine-tune on proprietary data, and decide when to upgrade. The trade-off is data residency and operational control against provider-managed simplicity.
"Frontier" is not a certification. No organization controls the designation. Vendors apply it to whatever they are selling this quarter, so treat "frontier performance" in any product brief with appropriate skepticism. The benchmarks that actually differentiate frontier from mid-tier are covered in Section 05. Those are the signal. Marketing copy is not.
The model you selected six months ago is mid-tier now. Build the harness so that swap is a config change, not a crisis.
Visa's Vulnerability and Vulnerability Agentic Hunter (VVAH) is a nine-stage agentic SAST pipeline released under Apache 2.0 and built on learnings from Anthropic's Project Glasswing. It is the most detailed public example of a production multi-agent harness doing real security work at scale.
The bottleneck in agentic security analysis is not finding vulnerabilities. Discovery accuracy is high even with older models. The bottleneck is triage speed: how fast can the system surface findings that require human attention without drowning analysts in false positives. The primary metric is MTTA, Mean Time to Adapt, not precision or recall alone.
The frontier model landscape changes faster than almost any technical field. The table below is a current snapshot, not a permanent ranking. What matters more than the specific numbers is understanding the categories and the trade-offs that govern harness design decisions.
| Model | Provider | Context | Weights | Modalities | License |
|---|---|---|---|---|---|
| Claude Opus 4 | Anthropic | 200K tokens | Closed | Text, vision | API only |
| GPT-4o | OpenAI | 128K tokens | Closed | Text, vision, audio | API only |
| Gemini 2.5 Pro | 1M tokens | Closed | Text, vision, audio, video | API only | |
| Llama 3.3 70B | Meta | 128K tokens | Open | Text | Llama 3 Community |
| Mistral Large 2 | Mistral AI | 128K tokens | Open | Text, vision | Apache 2.0 |
| DeepSeek R2 | DeepSeek | 64K tokens | Open | Text | MIT / API |
| Grok 3 | xAI | 131K tokens | Closed | Text, vision | API only |
The closed/open distinction matters for harness design in four ways: data residency, inference cost, customization depth, and supply chain risk.
A 1M token context window does not mean 1M tokens of useful working memory. The "lost in the middle" problem is well-documented: models retrieve information from the beginning and end of long contexts more reliably than from the middle. Critical instructions and retrieved facts belong at the top of the context. Supporting material goes last. Gemini 2.5's 1M window is a real capability advantage for long-document analysis, but routing strategy and context assembly discipline still determine whether that window is used effectively.
MMLU, HumanEval, and GSM8K are all at or above 90% for frontier models. They no longer differentiate. The field has moved on, but the press releases have not.
These are the benchmarks that still differentiate frontier models and inform real harness design decisions.
Selecting models based on MMLU scores is procurement theater. The benchmarks that still differentiate (HLE, GPQA Diamond, SWE-Bench, tau-bench) share a pattern: hard tasks need reasoning chains, tool use, and multi-step planning, not raw knowledge. The harness enables all three. A well-designed harness with a mid-tier model frequently beats a poorly designed harness with a frontier model.
OpenAI's o1 introduced the reasoning model pattern: give the model an internal scratchpad before it commits to an answer. Claude's extended thinking mode, DeepSeek R1, and others followed. Same mechanism, different pricing and latency profiles.
Claude Opus 4 (standard), GPT-4o, Gemini 2.5, Llama 3.3
o3, Claude extended thinking, DeepSeek R1
The harness routing layer determines when a task warrants a reasoning model. The signal is not task complexity in the abstract; it is whether the task has a structure where more thinking time demonstrably improves accuracy. Mathematical proofs, adversarial verification (as in VVAH S6), security threat modeling, and long-horizon planning are strong routing signals for reasoning models. Summarization, entity extraction, and code formatting are not. Getting this routing decision wrong in either direction is expensive: reasoning models on simple tasks waste latency and cost; completion models on hard tasks produce confident wrong answers.
Single-model deployments have one trust boundary: the user request. Multi-agent systems have one per hop. When an orchestrator delegates to a subagent, the subagent needs to know who is calling, what they are authorized to request, and whether they have been compromised upstream. Current protocols solve part of that problem. Not all of it.
Google's Agent-to-Agent (A2A) protocol, released in 2025, handles the identity layer. An Agent Card is a cryptographically signed document stating an agent's identity, capabilities, and authorized scope. When an orchestrator delegates to a subagent, it passes an Agent Card. The subagent verifies the card and checks whether the request fits the declared scope. That is a real improvement over nothing. It is not a complete solution.
A2A Agent Cards solve the identity assertion problem but leave a critical gap unaddressed: there is no published evaluation suite that tests whether delegated agents actually respect scope boundaries when the upstream delegating agent is adversarially manipulated. An orchestrator agent that has been compromised via prompt injection can present a valid, cryptographically authentic Agent Card while delegating actions that fall outside the original task scope. The receiving agent has no basis for detecting this because the credential is legitimate even if the intent is not.
Dual-Intent Runtime Authorization (DIRA) attacks the problem differently. Instead of trusting the credential, it requires the agent to declare its intent at task start. That declaration is cryptographically bound to the session. At execution time, DIRA compares actual tool calls and data access against the declared intent. Deviations generate audit events. Large enough deviations block execution.
In a multi-agent pipeline, DIRA gates run at each delegation hop, not just the entry point. A compromised orchestrator can pass a valid Agent Card and a manipulated intent. The downstream subagent's DIRA gate catches the mismatch between declared scope and actual execution request. The credential is not enough; the behavior has to match.
| Threat | A2A Agent Cards | DIRA |
|---|---|---|
| Impersonation of a legitimate agent | Addressed (cryptographic identity) | Addressed (session binding) |
| Scope creep via manipulated orchestrator | Not addressed | Addressed (intent vs. execution audit) |
| Indirect prompt injection via tool results | Not addressed | Partial (context drift detection) |
| Capability escalation across delegation hops | Partial (scope in Agent Card) | Addressed (per-hop gate) |
| Replay of a valid Agent Card for unauthorized task | Not addressed (nonce support varies) | Addressed (session-scoped binding) |
Enterprise AI projects fail at the harness layer, not the model layer. The system prompt, the retrieval strategy, the tool authorization design, the observability configuration: get any of those wrong and a frontier model produces garbage reliably.
An operating system abstracts hardware from applications. You write to OS APIs, not CPU instruction sets. When Intel ships a new architecture, your application does not get rewritten; the OS handles translation. Applications outlast CPUs because the OS absorbs the hardware churn.
A well-designed harness does the same thing for model churn. Keep your retrieval strategy, tool contracts, routing logic, and evaluation criteria at the harness layer rather than tied to a specific model's quirks. Then swapping models is an infrastructure config change, not a product redesign.
Organizations that chase model releases instead of investing in the harness layer will spend the next decade rewriting applications every six months. The ones that get durable value from AI are betting on the layer they control.
The harness is the product. The model is a swappable dependency you do not control.
Build your harness so that "swap the model" is a config change, not a code change. This means: no model-specific prompt formatting embedded in application code, no hardcoded model behaviors in business logic, evaluation criteria defined against observable outputs not model internals, and routing logic that treats each model as a capability provider behind a stable interface.
Wrong documents retrieved, retrieved facts placed in the middle of long contexts, stale memory injected from prior sessions, system prompt overwritten by adversarial user input. The model reasons correctly over incorrect input and produces confidently wrong output.
Tools registered in the function manifest that should not be available for a given task type or user role. The model calls a write API when it should be read-only. Data that should stay within a tenant boundary crosses into another session's context via shared retrieval indices.
No trace logging means no ability to reconstruct what the model saw when it produced a wrong or harmful output. No cost tracking means runaway token consumption goes undetected until the billing alert fires. No behavioral baseline means model degradation after a provider update is invisible until users report it.
System prompts that work in development break under production input distribution. Instructions that worked with one model version stop working after a provider update. No evaluation harness means regressions are caught by users, not by tests.
Long-term memory that accumulates errors over time with no correction mechanism. Episodic memory that retrieves irrelevant past sessions at high similarity scores because the retrieval model was not tuned for the task. Memory writes that race with reads across concurrent sessions and produce corrupted state.
Over-reliance on model-level safety filtering with no harness-level backstop. PII that flows through to model context without tokenization, then appears in log files. No intent classification gate, so adversarial prompts can escalate privileges by describing themselves as an authorized system role.
Answer five questions. Get a recommended starting level, the three sections to read first, and what to add when you are ready to go deeper.
What are you trying to evaluate?
How much ML/AI experience does your team have?
Are you evaluating a standalone model or an agentic system with tool use, multi-step planning, or autonomous action?
What is the consequence of a failure in your system?
Do you have an existing evaluation framework, or are you starting from scratch?
Check what your system needs. The right panel updates in real time: which harness patterns apply, what to prioritize, and a summary of your evaluation stack.
Check items on the left to see recommendations.