自定义中间件 (Custom middleware)
编写专属中间件,深度定制智能体各个阶段的拦截与处理逻辑
通过自定义中间件,你可以拦截智能体执行的每一个关键生命周期节点,修改输入提示词、动态注入工具、改写响应结果或追踪审计日志。
拦截钩子类型 (Hooks)
中间件支持两种维度的拦截钩子:
1. 节点级钩子 (Node-style hooks)
在特定的图节点执行前或执行后触发:
beforeAgent:在整个智能体开始前执行。afterAgent:在整个智能体执行完成、生成最终答复后执行。beforeModel:在向大模型发送请求前执行。afterModel:在模型生成响应后、解析工具调用前执行。beforeTools:在批量执行工具调用前执行。afterTools:在所有工具执行完毕后执行。
2. 包装级洋葱钩子 (Wrap-style hooks)
围绕核心操作形成“洋葱模型”包裹,支持类似 Express/Koa 中间件的 next() 链式调用:
wrapModelCall:环绕拦截单次模型调用,可在调用前修改请求、捕获异常或修改返回的生成对象。wrapToolCall:环绕拦截单个工具执行,可用于单工具计时、参数修改、错误降级或阻断。
import { createMiddleware, AIMessage } from "langchain";
const createMessageLimitMiddleware = (maxMessages: number = 50) => {
return createMiddleware({
name: "MessageLimitMiddleware",
beforeModel: {
canJumpTo: ["end"],
hook: (state) => {
if (state.messages.length === maxMessages) {
return {
messages: [new AIMessage("Conversation limit reached.")],
jumpTo: "end",
};
}
return;
}
},
afterModel: (state) => {
const lastMessage = state.messages[state.messages.length - 1];
console.log(`Model returned: ${lastMessage.content}`);
return;
},
});
};状态更新规范 (State updates)
中间件不仅能观察执行过程,还能向 LangGraph 状态图写入更新:
- Node 风格钩子:直接返回一个包含状态增量的对象(例如
return { messages: [newMsg] }),框架会自动将其与现有图状态合并。 - Wrap 风格钩子:通过修改请求入参或对响应结果进行管道式变换来影响下游状态。
多中间件的组合执行顺序 (Composition)
多个中间件按照在 middleware 数组中声明的先后顺序依次执行。对于 Wrap 风格钩子,越靠前的中间件位于洋葱模型的最外层。
import { createMiddleware } from "langchain";
const createRetryMiddleware = (maxRetries: number = 3) => {
return createMiddleware({
name: "RetryMiddleware",
wrapModelCall: (request, handler) => {
for (let attempt = 0; attempt < maxRetries; attempt++) {
try {
return handler(request);
} catch (e) {
if (attempt === maxRetries - 1) {
throw e;
}
console.log(`Retry ${attempt + 1}/${maxRetries} after error: ${e}`);
}
}
throw new Error("Unreachable");
},
});
};使用 createMiddleware 创建中间件
通过官方导出的 createMiddleware 函数,以强类型安全的方式定义中间件:
import { createMiddleware } from "langchain";
import * as z from "zod";
const trackingStateSchema = z.object({
modelCallCount: z.number().default(0),
});
const incrementAfterModel = createMiddleware({
name: "incrementAfterModel",
stateSchema: trackingStateSchema,
afterModel: (state) => {
return { modelCallCount: state.modelCallCount + 1 };
},
});扩展自定义状态 Schema (Custom state schema)
如果你的中间件需要保存业务专有的自定义状态字段(例如用户累计积分、审批单流转状态),可以通过扩展状态 Schema 来实现,所有字段都将在检查点中得到自动持久化。
import * as z from "zod";
import { createMiddleware } from "langchain";
import { Command } from "@langchain/langgraph";
const usageTrackingStateSchema = z.object({
lastModelCallTokens: z.number().optional(),
});
const trackUsage = createMiddleware({
name: "trackUsage",
stateSchema: usageTrackingStateSchema,
wrapModelCall: async (request, handler) => {
const response = await handler(request);
return new Command({ update: { lastModelCallTokens: 150 } });
},
});自定义流式数据转换器 (Stream transformers)
中间件还可以拦截流式传输事件,向前端插入自定义的计算进度通知或对敏感词执行流式实时替换。
访问运行时上下文 (Custom context)
中间件可以通过 request.runtime.context 访问每次调用时传入的租户 ID、认证用户与私有业务上下文。
智能体控制流跳转 (Agent jumps)
在特定紧急情况下(如检测到严重违规内容、余额不足),中间件可以直接返回特殊的跳转指令,短路后续所有的模型调用或工具执行,提前结束本次任务。
典型实战案例 (Examples)
1. 动态系统提示词注入 (Dynamic prompt)
根据当前用户等级动态追加人设要求:
import * as z from "zod";
import { createMiddleware } from "langchain";
import { Command, StateSchema, ReducedValue } from "@langchain/langgraph";
import { AIMessage, SystemMessage } from "@langchain/core/messages";
/** Last-wins reducer: when both middleware write, outer overwrites inner. */
const customMiddlewareStateSchema = new StateSchema({
traceLayer: new ReducedValue(
z.string().optional(),
{ reducer: (a, b) => b },
),
});
const outerMiddleware = createMiddleware({
name: "OuterMiddleware",
stateSchema: customMiddlewareStateSchema,
wrapModelCall: async (_request, handler) => {
await handler(_request);
return new Command({
update: {
traceLayer: "outer",
messages: [new SystemMessage({ content: "[Outer ran]" })],
},
});
},
});
const innerMiddleware = createMiddleware({
name: "InnerMiddleware",
stateSchema: customMiddlewareStateSchema,
wrapModelCall: async (_request, handler) => {
await handler(_request);
return new Command({
update: {
traceLayer: "inner",
messages: [new SystemMessage({ content: "[Inner ran]" })],
},
});
},
});2. 动态模型热替换 (Dynamic model selection)
根据对话复杂度或用户套餐动态切换底层模型:
from langchain.agents.middleware import (
AgentMiddleware,
AgentState,
ModelRequest,
ModelResponse,
)
from langgraph.runtime import Runtime
from typing import Any, Callable
class LoggingMiddleware(AgentMiddleware):
def before_model(self, state: AgentState, runtime: Runtime) -> dict[str, Any] | None:
print(f"About to call model with {len(state['messages'])} messages")
return None
def after_model(self, state: AgentState, runtime: Runtime) -> dict[str, Any] | None:
print(f"Model returned: {state['messages'][-1].content}")
return None
async def abefore_model(
self, state: AgentState, runtime: Runtime
) -> dict[str, Any] | None:
# Async version of before_model
return None
async def aafter_model(
self, state: AgentState, runtime: Runtime
) -> dict[str, Any] | None:
# Async version of after_model
print(f"Model returned: {state['messages'][-1].content}")
return None
agent = create_agent(
model="gpt-5.5",
middleware=[LoggingMiddleware()],
tools=[...],
)3. 工具调用审计监控 (Tool call monitoring)
记录每一个工具调用的入参、耗时与报错,上报至 APM 监控系统:
import { createMiddleware } from "langchain";
const loggingMiddleware = createMiddleware({
name: "LoggingMiddleware",
beforeModel: (state) => {
console.log(`About to call model with ${state.messages.length} messages`);
return;
},
afterModel: (state) => {
const lastMessage = state.messages[state.messages.length - 1];
console.log(`Model returned: ${lastMessage.content}`);
return;
},
});4. Anthropic 提示词缓存控制 (Prompt caching)
为静态的超长提示词添加缓存断点标记,节约高达 90% 的调用成本。
import { createMiddleware, createAgent, HumanMessage } from "langchain";
import { StateSchema } from "@langchain/langgraph";
import * as z from "zod";
const CustomState = new StateSchema({
modelCallCount: z.number().default(0),
userId: z.string().optional(),
});
const callCounterMiddleware = createMiddleware({
name: "CallCounterMiddleware",
stateSchema: CustomState,
beforeModel: {
canJumpTo: ["end"],
hook: (state) => {
if (state.modelCallCount > 10) {
return { jumpTo: "end" };
}
return;
},
},
afterModel: (state) => {
return { modelCallCount: state.modelCallCount + 1 };
},
});
const agent = createAgent({
model: "gpt-5.5",
tools: [...],
middleware: [callCounterMiddleware],
});
const result = await agent.invoke({
messages: [new HumanMessage("Hello")],
modelCallCount: 0,
userId: "user-123",
});State fields can be either public or private. Fields that start with an underscore (_) are considered private and will not be included in the agent's result. Only public fields (those without a leading underscore) are returned.
This is useful for storing internal middleware state that shouldn't be exposed to the caller, such as temporary tracking variables or internal flags:
import { StateSchema } from "@langchain/langgraph";
import * as z from "zod";
const PrivateState = new StateSchema({
// Public field - included in invoke result
publicCounter: z.number().default(0),
// Private field - excluded from invoke result
_internalFlag: z.boolean().default(false),
});
const middleware = createMiddleware({
name: "ExampleMiddleware",
stateSchema: PrivateState,
afterModel: (state) => {
// Both fields are accessible during execution
if (state._internalFlag) {
return { publicCounter: state.publicCounter + 1 };
}
return { _internalFlag: true };
},
});
const result = await agent.invoke({
messages: [new HumanMessage("Hello")],
publicCounter: 0
});
// result only contains publicCounter, not _internalFlag
console.log(result.publicCounter); // 1
console.log(result._internalFlag); // undefinedCustom stream transformers
NOTE
Middleware-registered transformers require langchain@1.4.3 or later.
Middleware can register stream transformer factories that project events from the live agent stream onto typed extension channels. This is useful for surfacing counters, side-channel artifacts, partial outputs, or wire-level redaction without coupling to the framework's built-in projections.
At compile time, middleware-registered factories merge with anything the caller passes directly to the agent factory. The final ordering rules keep the built-in ToolCallTransformer in front and let caller-supplied entries land last.
Pass streamTransformers to createMiddleware as a tuple of factories. Each factory has the shape () => StreamTransformer<any> (zero arguments) and is invoked once per scope; returning a fresh transformer per call keeps each subgraph isolated.
import { createAgent, createMiddleware } from "langchain";
const toolActivityMiddleware = createMiddleware({
name: "ToolActivityMiddleware",
streamTransformers: [toolActivityTransformer],
});
const agent = createAgent({
model: "gpt-5-nano",
tools: [...],
middleware: [toolActivityMiddleware],
});See Register transformers on middleware for the full ordering rules and the PII redaction example.
Custom context
Middleware can define a custom context schema to access per-invocation metadata. Unlike state, context is read-only and not persisted between invocations. This makes it ideal for:
- User information: Pass user ID, roles, or preferences that don't change during execution
- Configuration overrides: Provide per-invocation settings like rate limits or feature flags
- Tenant/workspace context: Include organization-specific data for multi-tenant applications
- Request metadata: Pass request IDs, API keys, or other metadata needed by middleware
Define a context schema using Zod and access it via runtime.context in middleware hooks. Required fields in the context schema will be enforced at the TypeScript level, ensuring you must provide them when calling agent.invoke().
import { createAgent, createMiddleware, HumanMessage } from "langchain";
import * as z from "zod";
const contextSchema = z.object({
userId: z.string(),
tenantId: z.string(),
apiKey: z.string().optional(),
});
const userContextMiddleware = createMiddleware({
name: "UserContextMiddleware",
contextSchema,
wrapModelCall: (request, handler) => {
// Access context from runtime
const { userId, tenantId } = request.runtime.context;
// Add user context to system message
const contextText = `User ID: ${userId}, Tenant: ${tenantId}`;
const newSystemMessage = request.systemMessage.concat(contextText);
return handler({
...request,
systemMessage: newSystemMessage,
});
},
});
const agent = createAgent({
model: "gpt-5.5",
middleware: [userContextMiddleware],
tools: [],
contextSchema,
});
const result = await agent.invoke(
{ messages: [new HumanMessage("Hello")] },
// Required fields (userId, tenantId) must be provided
{
context: {
userId: "user-123",
tenantId: "acme-corp",
},
}
);Required context fields: When you define required fields in your contextSchema (fields without .optional() or .default()), TypeScript will enforce that these fields must be provided during agent.invoke() calls. This ensures type safety and prevents runtime errors from missing required context.
// This will cause a TypeScript error if userId or tenantId are missing
const result = await agent.invoke(
{ messages: [new HumanMessage("Hello")] },
{ context: { userId: "user-123" } } // Error: tenantId is required
);Execution order
When using multiple middleware, understand how they execute:
const agent = createAgent({
model: "gpt-5.5",
middleware: [middleware1, middleware2, middleware3],
tools: [...],
});Execution flow
Before hooks run in order:
middleware1.before_agent()middleware2.before_agent()middleware3.before_agent()Agent loop starts
middleware1.before_model()middleware2.before_model()middleware3.before_model()Wrap hooks nest like function calls:
middleware1.wrap_model_call()→middleware2.wrap_model_call()→middleware3.wrap_model_call()→ modelAfter hooks run in reverse order:
middleware3.after_model()middleware2.after_model()middleware1.after_model()Agent loop ends
middleware3.after_agent()middleware2.after_agent()middleware1.after_agent()
Key rules:
before_*hooks: First to lastafter_*hooks: Last to first (reverse)wrap_*hooks: Nested (first middleware wraps all others)
Agent jumps
To exit early from middleware, return a dictionary with jump_to:
Available jump targets:
'end': Jump to the end of the agent execution (or the firstafter_agenthook)'tools': Jump to the tools node'model': Jump to the model node (or the firstbefore_modelhook)
import { createAgent, createMiddleware, AIMessage } from "langchain";
const agent = createAgent({
model: "gpt-5.5",
middleware: [
createMiddleware({
name: "BlockedContentMiddleware",
beforeModel: {
canJumpTo: ["end"],
hook: (state) => {
if (state.messages.at(-1)?.content.includes("BLOCKED")) {
return {
messages: [new AIMessage("I cannot respond to that request.")],
jumpTo: "end" as const,
};
}
return;
},
},
}),
],
});
const result = await agent.invoke({
messages: "Hello, world! BLOCKED"
});
/**
* Expected output:
* I cannot respond to that request.
*/
console.log(result.messages.at(-1)?.content);Best practices
- Keep middleware focused - each should do one thing well
- Handle errors gracefully - don't let middleware errors crash the agent
- Use appropriate hook types:
- Node-style for sequential logic (logging, validation)
- Wrap-style for control flow (retry, fallback, caching)
- Clearly document any custom state properties
- Unit test middleware independently before integrating
- Consider execution order - place critical middleware first in the list
- Use built-in middleware when possible
Examples
Dynamic prompt
Dynamically modify the system prompt at runtime to inject context, user-specific instructions, or other information before each model call. This is one of the most common middleware use cases.
Use the systemMessage field in ModelRequest to read and modify the system prompt. It contains a SystemMessage object (even if the agent was created with a string systemPrompt).
import { createMiddleware, SystemMessage, createAgent } from "langchain";
const addContextMiddleware = createMiddleware({
name: "AddContextMiddleware",
wrapModelCall: async (request, handler) => {
return handler({
...request,
systemMessage: request.systemMessage.concat(`Additional context.`),
});
},
});
const agent = createAgent({
model: "google:gemini-3.6-flash",
systemPrompt: "You are a helpful assistant.",
middleware: [addContextMiddleware],
});import { createMiddleware, SystemMessage, createAgent } from "langchain";
const addContextMiddleware = createMiddleware({
name: "AddContextMiddleware",
wrapModelCall: async (request, handler) => {
return handler({
...request,
systemMessage: request.systemMessage.concat(`Additional context.`),
});
},
});
const agent = createAgent({
model: "openai:gpt-5.5",
systemPrompt: "You are a helpful assistant.",
middleware: [addContextMiddleware],
});import { createMiddleware, SystemMessage, createAgent } from "langchain";
const addContextMiddleware = createMiddleware({
name: "AddContextMiddleware",
wrapModelCall: async (request, handler) => {
return handler({
...request,
systemMessage: request.systemMessage.concat(`Additional context.`),
});
},
});
const agent = createAgent({
model: "anthropic:claude-sonnet-5",
systemPrompt: "You are a helpful assistant.",
middleware: [addContextMiddleware],
});import { createMiddleware, SystemMessage, createAgent } from "langchain";
const addContextMiddleware = createMiddleware({
name: "AddContextMiddleware",
wrapModelCall: async (request, handler) => {
return handler({
...request,
systemMessage: request.systemMessage.concat(`Additional context.`),
});
},
});
const agent = createAgent({
model: "openrouter:z-ai/glm-5.2",
systemPrompt: "You are a helpful assistant.",
middleware: [addContextMiddleware],
});import { createMiddleware, SystemMessage, createAgent } from "langchain";
const addContextMiddleware = createMiddleware({
name: "AddContextMiddleware",
wrapModelCall: async (request, handler) => {
return handler({
...request,
systemMessage: request.systemMessage.concat(`Additional context.`),
});
},
});
const agent = createAgent({
model: "fireworks:accounts/fireworks/models/glm-5p2",
systemPrompt: "You are a helpful assistant.",
middleware: [addContextMiddleware],
});import { createMiddleware, SystemMessage, createAgent } from "langchain";
const addContextMiddleware = createMiddleware({
name: "AddContextMiddleware",
wrapModelCall: async (request, handler) => {
return handler({
...request,
systemMessage: request.systemMessage.concat(`Additional context.`),
});
},
});
const agent = createAgent({
model: "baseten:zai-org/GLM-5.2",
systemPrompt: "You are a helpful assistant.",
middleware: [addContextMiddleware],
});import { createMiddleware, SystemMessage, createAgent } from "langchain";
const addContextMiddleware = createMiddleware({
name: "AddContextMiddleware",
wrapModelCall: async (request, handler) => {
return handler({
...request,
systemMessage: request.systemMessage.concat(`Additional context.`),
});
},
});
const agent = createAgent({
model: "ollama:north-mini-code-1.0",
systemPrompt: "You are a helpful assistant.",
middleware: [addContextMiddleware],
});Use SystemMessage.concat to preserve cache control metadata or structured content blocks created by other middleware.
Dynamic model selection
import { createMiddleware, initChatModel } from "langchain";
const models = {
complex: await initChatModel("claude-sonnet-4-6"),
simple: await initChatModel("claude-haiku-4-5-20251001"),
};
const dynamicModelMiddleware = createMiddleware({
name: "DynamicModelMiddleware",
wrapModelCall: (request, handler) => {
const modifiedRequest = { ...request };
if (request.messages.length > 10) {
modifiedRequest.model = models.complex;
} else {
modifiedRequest.model = models.simple;
}
return handler(modifiedRequest);
},
});Dynamically selecting tools
Select relevant tools at runtime to improve performance and accuracy. This section covers filtering pre-registered tools. For registering tools that are discovered at runtime (e.g., from MCP servers), see Runtime tool registration.
Benefits:
- Shorter prompts - Reduce complexity by exposing only relevant tools
- Better accuracy - Models choose correctly from fewer options
- Permission control - Dynamically filter tools based on user access
import { createAgent, createMiddleware } from "langchain";
const toolSelectorMiddleware = createMiddleware({
name: "ToolSelector",
wrapModelCall: (request, handler) => {
// Select a small, relevant subset of tools based on state/context
const relevantTools = selectRelevantTools(request.state, request.runtime);
const modifiedRequest = { ...request, tools: relevantTools };
return handler(modifiedRequest);
},
});
const agent = createAgent({
model: "gpt-5.5",
tools: allTools,
middleware: [toolSelectorMiddleware],
});Tool call monitoring
import { createMiddleware } from "langchain";
const toolMonitoringMiddleware = createMiddleware({
name: "ToolMonitoringMiddleware",
wrapToolCall: (request, handler) => {
console.log(`Executing tool: ${request.toolCall.name}`);
console.log(`Arguments: ${JSON.stringify(request.toolCall.args)}`);
try {
const result = handler(request);
console.log("Tool completed successfully");
return result;
} catch (e) {
console.log(`Tool failed: ${e}`);
throw e;
}
},
});Prompt caching (Anthropic)
When working with Anthropic models, use structured content blocks with cache control directives to cache large system prompts:
Decorator
from langchain.agents.middleware import wrap_model_call, ModelRequest, ModelResponse
from langchain.messages import SystemMessage
from typing import Callable
@wrap_model_call
def add_cached_context(
request: ModelRequest,
handler: Callable[[ModelRequest], ModelResponse],
) -> ModelResponse:
# Always work with content blocks
new_content = list(request.system_message.content_blocks) + [
{
"type": "text",
"text": "Here is a large document to analyze:\n\n<document>...</document>",
# content up until this point is cached
"cache_control": {"type": "ephemeral"}
}
]
new_system_message = SystemMessage(content=new_content)
return handler(request.override(system_message=new_system_message))Class
from langchain.agents.middleware import AgentMiddleware, ModelRequest, ModelResponse
from langchain.messages import SystemMessage
from typing import Callable
class CachedContextMiddleware(AgentMiddleware):
def wrap_model_call(
self,
request: ModelRequest,
handler: Callable[[ModelRequest], ModelResponse],
) -> ModelResponse:
# Always work with content blocks
new_content = list(request.system_message.content_blocks) + [
{
"type": "text",
"text": "Here is a large document to analyze:\n\n<document>...</document>",
"cache_control": {"type": "ephemeral"} # This content will be cached
}
]
new_system_message = SystemMessage(content=new_content)
return handler(request.override(system_message=new_system_message))Notes:
ModelRequest.system_messageis always aSystemMessageobject, even if the agent was created withsystem_prompt="string"- Use
SystemMessage.content_blocksto access content as a list of blocks, regardless of whether the original content was a string or list - When modifying system messages, use
content_blocksand append new blocks to preserve existing structure - You can pass
SystemMessageobjects directly tocreate_agent'ssystem_promptparameter for advanced use cases like cache control
:::
Modify system messages in middleware using the systemMessage field in ModelRequest. It contains a SystemMessage object (even if the agent was created with a string systemPrompt).
Example: Chaining middleware - Different middleware can use different approaches:
import { createMiddleware, SystemMessage, createAgent } from "langchain";
// Middleware 1: Uses systemMessage with simple concatenation
const myMiddleware = createMiddleware({
name: "MyMiddleware",
wrapModelCall: async (request, handler) => {
return handler({
...request,
systemMessage: request.systemMessage.concat(`Additional context.`),
});
},
});
// Middleware 2: Uses systemMessage with structured content (preserves structure)
const myOtherMiddleware = createMiddleware({
name: "MyOtherMiddleware",
wrapModelCall: async (request, handler) => {
return handler({
...request,
systemMessage: request.systemMessage.concat(
new SystemMessage({
content: [
{
type: "text",
text: " More additional context. This will be cached.",
cache_control: { type: "ephemeral", ttl: "5m" },
},
],
})
),
});
},
});
const agent = createAgent({
model: "google:gemini-3.6-flash",
systemPrompt: "You are a helpful assistant.",
middleware: [myMiddleware, myOtherMiddleware],
});The resulting system message will be:
new SystemMessage({
content: [
{ type: "text", text: "You are a helpful assistant." },
{ type: "text", text: "Additional context." },
{
type: "text",
text: " More additional context. This will be cached.",
cache_control: { type: "ephemeral", ttl: "5m" },
},
],
});Use SystemMessage.concat to preserve cache control metadata or structured content blocks created by other middleware.
Additional resources
Connect these docs to your agent of choice via MCP for real-time answers.
