here<\/a>.<\/p>\n1.2 Defining The Agent<\/p>\n
Next, we configure the agent. For that, we use the OpenAI Agents SDK. <\/p>\n
At a high level, the agent is just this:<\/p>\n
# pip install openai-agents
\nfrom agents import Agent<\/p>\n
agent = Agent(
\n name=”Policy research assistant”,
\n instructions=INSTRUCTIONS,
\n model=”gpt-5.4″,
\n tools=[list_docs, search_docs, read_doc],
\n)<\/p>\n
Two parts we need to go through: the agent instruction, and the tools it has access to.<\/p>\n
First, the instruction. This is where we define the desired search behavior:<\/p>\n
# Note: This instruction is iterated with AI
\nINSTRUCTIONS = “””
\n[Role]
\nYou are a careful internal policy research assistant.<\/p>\n
[Research behavior]
\nAnswer employee policy questions using the document tools.
\nFind enough relevant evidence to support the answer.
\nKeep conclusions grounded in the policy documents.<\/p>\n
[Expected output]
\nGive a direct answer first.
\nThen briefly explain the evidence.
\nCite the document filenames used for each important claim.
\n“””.strip()<\/p>\n
For this case study, we require that the agent can only touch the docs via three pre-defined tools:<\/p>\n
The first one is a tool that gives the agent a quick overview of what documents exist:<\/p>\n
@function_tool
\ndef list_docs() -> list[dict]:
\n “””List available policy documents without returning their body text.”””
\n return [
\n {
\n “doc_name”: doc[“doc_name”],
\n “title”: doc[“title”],
\n “effective”: doc[“effective”],
\n “summary”: doc[“summary”],
\n }
\n for doc in docs.values()
\n ]<\/p>\n
The second tool is a keyword-search tool. We keep it simple here: each document is split into paragraph chunks, and each query is matched against those chunks by token overlap:<\/p>\n
@function_tool
\ndef search_docs(query: str) -> list[dict]:
\n “””Search policy documents and return the top three short snippets.”””
\n query_tokens = tokenize(query)
\n scored = []<\/p>\n
for chunk in chunks:
\n score = len(query_tokens & chunk[“tokens”])
\n if score:
\n scored.append((score, chunk))<\/p>\n
scored.sort(key=lambda item: item[0], reverse=True)<\/p>\n
results = []
\n for score, chunk in scored[:3]:
\n snippet = chunk[“text”].replace(“\\n”, ” “)
\n if len(snippet) > 420:
\n snippet = snippet[:417].rstrip() + “…”
\n results.append({
\n “doc_name”: chunk[“doc_name”],
\n “title”: chunk[“title”],
\n “section”: chunk[“section”],
\n “snippet”: snippet,
\n “score”: round(score, 2),
\n })<\/p>\n
return results<\/p>\n
The last tool is what allows the agent to open one document by filename:<\/p>\n
@function_tool
\ndef read_doc(doc_name: str) -> str:
\n “””Read one policy document by filename.”””
\n if doc_name not in docs:
\n valid = “, “.join(sorted(docs))
\n return f”Unknown document: {doc_name}. Valid documents: {valid}”<\/p>\n
return docs[doc_name][“text”]<\/p>\n
That\u2019s the full RAG agent.<\/p>\n
1.3 Running One Policy Question<\/p>\n
Now we test the agent with one concrete question:<\/p>\n
\u201cI am attending a conference in Berlin. The conference organizer lists an official hotel, but the nightly rate is above the normal hotel cap. Can I book that hotel, and what approval do I need before booking?\u201c<\/p>\n
We run the agent with:<\/p>\n
from agents import Runner<\/p>\n
result = await Runner.run(agent, PROMPT, max_turns=12)<\/p>\n
The agent produced the right answer: yes, the employee can book the official conference hotel if there is a practical business reason. It got that information from conference_guidelines.md.<\/p>\n
For the approval part, the agent first identified that approval is needed as the hotel is above the normal cap. Then it gave the corresponding approval conditions. The agent used travel_policy.md, approval_matrix.md, and policy_updates_2026.md to support its answer, which is exactly what we would expect.<\/p>\n
The more interesting part is the trace, from which we can learn how the agent thinks. We can show the trace in the following way:<\/p>\n
for item in result.new_items:
\n print(type(item).__name__, item)<\/p>\n
result.new_items contains the intermediate tool calls and tool outputs produced by the agent. In my run, I can see that the agent first called search_docs() with keywords like conference hotel, hotel cap, approval, and Berlin. Then, it called list_docs() to inspect the available policy documents. After that, it opened the relevant files with read_doc(). Only then did it produce the final answer.<\/p>\n
This is exactly the agentic loop we wanted to see.<\/p>\n
3. What to Decide Before Building Agentic RAG<\/p>\n
The case study we just went through only scratched the surface. To really build a practical agentic RAG solution, based on my experience, I suggest you answer the following 5 questions:<\/p>\n
Q1: How much freedom should the agent have?<\/p>\n
One common option is exactly what we have done in the earlier case study: we exposed a couple of carefully curated tools, and the agent is only allowed to use those tools to do the investigation. This is straightforward in terms of controlling, testing, and auditing.<\/p>\n
But we can also give the agent broader access, such as shell and file system. This way, the agent can directly run scripts to search and inspect files, and maybe even do further data processing to generate useful artifacts, all on its own. <\/p>\n
This pattern can be much more powerful, but it also increases risk and makes behavior harder to predict.<\/p>\n
So for most RAG applications, I\u2019d start with curated tools first, and only add shell\/file-system access when the task complexity justifies it.<\/p>\n
Q2: Should the agent search raw text only?<\/p>\n
Most RAG projects might start with plain text like PDFs, wiki pages, manuals, etc. That\u2019s fine.<\/p>\n
But in practice, we can often make retrieval easier by deriving a knowledge layer on top of the raw texts.<\/p>\n
Those derived knowledge artifacts can be document metadata, summaries, cross-document links, or we can go further and implement a proper knowledge graph.<\/p>\n
These derived knowledge artifacts help the agent navigate the corpus, while the raw texts remain as the source of truth.<\/p>\n
Q3: Do we still need embeddings?<\/p>\n
Agentic RAG doesn\u2019t necessarily mean embeddings are gone.<\/p>\n
Vector embeddings are still an efficient way to find semantically relevant texts, and it often outperforms a pure keyword search strategy. <\/p>\n
In agentic RAG, what changed essentially is that the retrieval becomes an \u201caction\u201d the agent can take. Under this framing, \u201caction\u201d can still be powered by an embedding-based retriever, a keyword-based one, or even a hybrid one.<\/p>\n
So embeddings can still be useful. They are just one possible way to power the agent\u2019s search tool.<\/p>\n
Q4: Should one agent handle everything?<\/p>\n
The simplest agentic RAG setup is just one agent that does the search, read, and answer.<\/p>\n
But as the task gets more complex, you might want to split the work among multiple agents. More concretely, you might need to adopt a multi-agent strategy.<\/p>\n
You can split the work by role. For example, the planner-retriever-writer split, where the planner decides what evidence is needed, the retriever collects it, and the writer produces the final answer by using the collected evidence.<\/p>\n
You can also split by source type, where each agent is equipped with customized tools and focuses on one specific type of source.<\/p>\n
Just keep in mind: A multi-agent setup adds coordination complexity, and there is no guarantee that it will perform better than a single-agent setup. Empirical testing is very important.<\/p>\n
Q5: Should we always use agentic RAG?<\/p>\n
Maybe not always.<\/p>\n
Just because agentic RAG becomes a trendy topic does not necessarily mean you should always default to it.<\/p>\n
Agentic RAG gives more flexibility, but that comes with costs. That cost is not only about latency or token cost, but also less predictable agent behavior.<\/p>\n
Always start simple, then add agentic loops when the question actually needs iterative retrieval.<\/p>\n","protected":false},"excerpt":{"rendered":"application we build is a RAG app. The recipe is simple: chunk, embed, retrieve, then answer. It looks…\n","protected":false},"author":2,"featured_media":104496,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[179,7493,54196,24,405,415,30539],"class_list":["post-104495","post","type-post","status-publish","format-standard","has-post-thumbnail","category-agentic-ai","tag-agentic-ai","tag-agentic-artificial-intelligence","tag-agentic-rag","tag-ai","tag-ai-agents","tag-llm","tag-rag"],"_links":{"self":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts\/104495","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/comments?post=104495"}],"version-history":[{"count":0,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts\/104495\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/media\/104496"}],"wp:attachment":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/media?parent=104495"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/categories?post=104495"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/tags?post=104495"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}