How To Build Deep Research Agents With GPT-5.6 and Evals
Learn Pratik BhavsarKey takeaways
- Divide broad research objectives into verifiable sub-tasks: Use a plan-and-execute architecture to decompose complex research questions into granular, sequential steps, which allows the agent to maintain focus and reduces the probability of systemic reasoning errors.
- Evaluate reasoning trajectories, not just final outcomes: Because an agent can reach a correct final answer via an unsafe or inefficient path, engineering teams must instrument every planning, tool-calling, and synthesis step to isolate where the reasoning process actually broke down.
- Balance model capability with operational cost: When building research agents that perform repeated, multi-step calls, select model configurations—such as gpt-5.6-terra—that optimize for the specific tradeoff between high-quality reasoning and the multiplied cost of iterative search calls.
Your research agent returns a page of sources and one confident number nobody can trace to a document. A deep research agent breaks a broad question into granular sub-questions, searches the web across multiple iterations, and synthesizes a sourced answer. The Deep Research launch occurred on February 2, 2025, and Google and Anthropic also ship hosted research systems. So, why build your own? Hosted systems limit control over research scope, tool use, and output evaluation.
The Gartner agent forecast predicts over 40% of agentic AI projects will be canceled by the end of 2027 because of escalating costs, unclear business value, or inadequate risk controls. Eval data is where all three problems show up first.
This tutorial builds a financial deep research agent using plan-and-execute in LangGraph, with gpt-5.6-terra for planning and generation, Tavily for web search, and Splunk Agent Observability for quality analysis. It also covers LangGraph, Tavily, and OpenAI application programming interface (API) changes that break older code.
Understanding why a custom deep research agent still needs evals
An aggregate success rate won't reveal whether planning, retrieval, or synthesis broke. The MAST study measured failure rates of 41%–86.7% across seven state-of-the-art open-source multi-agent systems and identified 14 failure-mode categories. The Stanford AI Index reports that autonomous agents still fail roughly one in three attempts on structured benchmarks. Only granular, span-level insight isolates the cause.
Architecture matters too — Anthropic's production research system uses an orchestrator-worker pattern, with Claude Opus 4 as lead and Claude Sonnet 4 subagents. The Anthropic research eval found it outperformed single-agent Claude Opus 4 by 90.2%. We use a simpler plan-and-execute loop.
If your system picks poor sources, repeats searches, or invents claims, every run burns budget without dependable insight. Evals must inspect both the final answer and the path that produced it.
Fixing LangGraph, Tavily, and OpenAI breaking changes
Three breaking changes can stop the build:
Deprecated: create_react_agent. The LangGraph migration guide states: "LangGraph v1 deprecates the create_react_agent prebuilt. Use LangChain's create_agent, which runs on LangGraph and adds a flexible middleware system." The old state_modifier parameter, later renamed prompt, is now system_prompt.
Deprecated: TavilySearchResults. Tavily documentation says the langchain_community import "remains functional for now" but strongly recommends the new langchain-tavily package, which exports TavilySearch.
Retired: The o-series models used in earlier builds. OpenAI's deprecation schedule shuts down o3-2025-04-16 and gpt-5-2025-08-07 on December 11, 2026, naming gpt-5.6-sol as the replacement. OpenAI shuts down o4-mini-2025-04-16 on October 23, 2026.
The legacy promptquality package and GalileoPromptCallback are superseded by the current eval software development kit (SDK); the callback below comes from the galileo package at the pinned version.
Choosing models for generation and planning
All three GPT-5.6 variants share a 1,050,000-token context window, 128,000-token maximum output, and configurable reasoning effort. Per-million-token pricing:
We use gpt-5.6-terra for planning and execution because repeated calls multiply cost. Before standardizing, compare at least two configurations on answer quality, tokens, successful tool calls, latency, and completion rate.
Installing dependencies and configuring API keys
Install the tested package versions:
Save your Tavily, OpenAI, and eval credentials in .env:
OPENAI_API_KEY=KKK
TAVILY_API_KEY=KKK
SPLUNK_AO_API_KEY=KKK
SPLUNK_AO_PROJECT=deep-research-agent
SPLUNK_AO_AGENT_STREAM=dev
Keep the file outside version control.
Building the deep research agent with plan-and-execute
The agent receives a question, creates sub-questions, searches with Tavily, analyzes results, and replans until the goal is achieved.
Defining the research agent and search tool
Create agent.ipynb and define the worker:
from langchain.agents import create_agent
from langchain_tavily import TavilySearch
system_prompt = (
"You are a helpful finance expert named Fred. First, create a plan to "
"answer the research query. Then use tools to answer each question in "
"the plan. Finally, use those answers to give your final verdict."
)
tools = [TavilySearch(max_results=10, topic="finance")]
agent_executor = create_agent(
"openai:gpt-5.6-terra", tools, system_prompt=system_prompt
)
The topic parameter accepts general, news, and finance; finance reduces irrelevant retrieval before results reach the model.
Validating retrieval quality
Test this node independently. Inspect returned URLs and confirm the final message contains evidence rather than unsupported conclusions.
Tracking state across iterations
The state stores the plan, completed steps and results, original input, and final response.
import operator
from typing import Annotated, List, Tuple
from typing_extensions import TypedDict
from pydantic import BaseModel, Field
class PlanExecute(TypedDict):
input: str
plan: List[str]
past_steps: Annotated[List[Tuple], operator.add]
response: str
class Plan(BaseModel):
"""Plan to follow in future"""
steps: List[str] = Field(
description="different steps to follow, should be in sorted order"
)
The Annotated[..., operator.add reducer appends results instead of overwriting them.
Testing state updates
Use two synthetic steps to confirm both tuples remain in past_steps and a new plan correctly replaces the previous plan.
Generating the research plan with structured output
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI
planner_prompt = ChatPromptTemplate.from_messages(
[
(
"system",
"You are a finance research agent. For the given objective, "
"come up with a simple step-by-step plan. This plan should "
"involve individual tasks, that if executed correctly will "
"yield the correct answer. Do not add any superfluous steps. "
"The result of the final step should be the final answer. "
"Make sure that each step has all the information needed - "
"do not skip steps.",
),
("placeholder", "{messages}"),
]
)
planner = planner_prompt | ChatOpenAI(
model="gpt-5.6-terra").with_structured_output(Plan)
Reviewing plan quality
Good steps are independently searchable, logically ordered, and specific enough to evaluate. Replace vague tasks such as "research Tesla" with bounded questions about deliveries, margins, or competition.
Defining replanning output
The replanner either creates new steps or returns a final answer:
from typing import Union
class Response(BaseModel):
"""Response to user."""
response: str
class Act(BaseModel):
"""Action to perform."""
action: Union[Response, Plan] = Field(
description="Action to perform. If you want to respond to user, use Response. "
"If you need to further use tools to get the answer, use Plan."
)
Creating the replanning prompt
replanner_prompt = ChatPromptTemplate.from_template(
"""For the given objective, come up with a simple step-by-step plan. \
This plan should involve individual tasks, that if executed correctly will \
yield the correct answer. Do not add any superfluous steps. The result of \
the final step should be the final answer. Make sure that each step has all \
the information needed - do not skip steps.
Your objective was this:
{input}
Your original plan was this:
{plan}
You have currently completed the following steps:
{past_steps}
Update your plan accordingly. If no more steps are needed and you can return \
to the user, then respond with that. Otherwise, fill out the plan. Only add \
steps to the plan that still NEED to be done. Do not return previously done \
steps as part of the plan."""
)
replanner = replanner_prompt | ChatOpenAI(
model="gpt-5.6-terra").with_structured_output(Act)
If completed work reappears, add explicit completion criteria and set a recursion limit.
Defining graph functions
from langgraph.graph import StateGraph, START, END
async def execute_step(state: PlanExecute):
plan = state["plan"]
plan_str = "\n".join(f"{i+1}. {step}" for i, step in enumerate(plan))
task = plan[0]
task_formatted = f"""For the following plan: {plan_str}\n\nYou are tasked with executing step {1}, {task}."""
agent_response = await agent_executor.ainvoke(
{"messages": [("user", task_formatted)]}
)
return {
"past_steps": [(task, agent_response["messages"][-1].content)],
}
async def plan_step(state: PlanExecute):
plan = await planner.ainvoke({"messages": [("user", state["input"])]})
return {"plan": plan.steps}
Compiling the workflow
async def replan_step(state: PlanExecute):
output = await replanner.ainvoke(state)
if isinstance(output.action, Response):
return {"response": output.action.response}
else:
return {"plan": output.action.steps}
def should_end(state: PlanExecute):
if "response" in state and state["response"]:
return END
else:
return "agent"
workflow = StateGraph(PlanExecute)
workflow.add_node("planner", plan_step)
workflow.add_node("agent", execute_step)
workflow.add_node("replan", replan_step)
workflow.add_edge(START, "planner")
workflow.add_edge("planner", "agent")
workflow.add_edge("agent", "replan")
workflow.add_conditional_edges("replan", should_end, ["agent", END])
app = workflow.compile()
Visualize it with display(Image(app.get_graph(xray=True).draw_mermaid_png())). Test both termination paths, an empty plan, a failed tool call, and a malformed response.
Evaluating your deep research agent
A completed run may still contain weak work. Evals must reveal whether the system chose appropriate tools, stayed grounded, and completed the objective.
Configuring agentic metrics on your log stream
Splunk Agent Observability organizes logs into sessions, traces, and spans. The relevant agent eval metrics answer different questions:
- Tool Selection Quality checks whether the system selected the correct tool and arguments.
- Context Adherence identifies information missing from retrieved context. A value near one indicates grounded output.
- Action Completion checks whether the system accomplished every goal in a session.
- Tool Error detection flags failures, exceptions, and unexpected tool behavior.
Prioritizing evaluation metrics
For financial research, prioritize grounding and tool choice. Review successful and failed runs manually to confirm each metric identifies behavior your team would change.
Instrumenting the run with one callback
from splunk_ao.handlers.langchain import SplunkAOAsyncCallback
inputs = {"input": "Should we invest in Tesla given the current situation of electric vehicles?"}
config = {"recursion_limit": 40, "callbacks": SplunkAOAsyncCallback()]}
async for event in app.astream(inputs, config=config):
for k, v in event.items():
if k != "__end__":
print(v)
SplunkAOAsyncCallback uses the flush_on_chain_end=True default, so telemetry is flushed when the graph finishes. The loop's print(v) call writes graph events to the local console.
Validating trace coverage
Run an ambiguous objective and confirm the trace exposes the plan, search invocations, returned context, replanning decision, and final response.
Choosing between large language model (LLM) judges and Luna evaluation models
Context Adherence Plus is powered by ChainPoll research (originally from Galileo), which prompts a model multiple times and aggregates boolean judgments. ChainPoll scored an aggregate area under the receiver operating characteristic curve (AUROC) of 0.78 on the RealHall benchmark, beating industry standards for LLMs by over 23%. Untuned judges show only 66%–68% agreement with human experts, and 93% of teams using LLM-as-a-judge report reliability problems.
For production-scale evals, Splunk Agent Observability's Luna evaluation models are purpose-built small language models (SLMs) with three billion and eight billion parameters, unrelated to OpenAI's gpt-5.6-luna.
Use judges for development diagnosis, then consider lower-latency variants labeled (SLM) for production coverage.
Debugging low context adherence and tool selection failures
When Context Adherence is low, its explanation identifies unsupported claims. On a Tesla run, recent quarterly figures may trace to search results while older quarters lack matching documents.
Improve retrieval: Use Tavily's time_range or start_date parameters to retrieve historical coverage.
Tighten the prompt: Require citations for numerical claims, distinguish verified from unverified data, and check completeness.
Tool Selection Quality explanations identify incorrect tools or arguments and can become added prompt criteria. For slow traces, Trace View pinpoints delays, Graph View renders branches and tool calls, and Splunk Agent Observability's Signals capability analyzes traces for security leaks, policy drift, and cascading failures.
Building reliable research workflows through evals
Reliable research comes from bounded tasks, retained state, controlled replanning, validated termination, and evidence checks. Splunk's mission has always been to provide visibility and insights that keep digital systems secure and reliable, and that mission now covers the agents running your business. Splunk Agent Observability is the intelligence layer for trusted agentic operations:
- Graph and Trace views: See decision paths, tool calls, latency, and bottlenecks.
- Agentic metrics: Measure tool selection, completion, efficiency, flow, and reasoning quality.
- Luna evaluation SLMs: Run lower-cost, low-latency production evals.
Read The Agentic Shift: Redefining Observability for the AI Era to see how observability must change for agentic systems before your agents reach production.
FAQs about deep research agents
Related Articles
15 Must-Have SIEM Features for Modern Threat Defense in 2026
The Digital Immune System (DIS) Explained
