Agent Foundry
All Problems

#22. Tool Fallback Strategy

MediumTool CallingError Recovery

The Problem

Your research assistant agent uses a primary_search tool to find information. However, this tool is unreliable — it intermittently throws ConnectionError exceptions. When it fails, the agent crashes and the user gets no results. A backup_search tool exists in the codebase but is never used. Your job is to implement a fallback strategy so the agent automatically tries backup_search when primary_search fails, ensuring the user always gets results.

Examples

Example 1

User input: Find recent research on transformer architectures

Current (bad) output: Half the time, the agent crashes with ConnectionError: Primary search service is temporarily unavailable and returns nothing.

Expected (good) output: If primary_search fails, the agent seamlessly falls back to backup_search and responds: I found these results from the backup search: Blog post about transformer architectures, Wikipedia article on transformer architectures.

Example 2

User input: What are the latest developments in quantum computing?

Current (bad) output: When the primary search works, results are fine. When it doesn't, the user sees an error traceback.

Expected (good) output: The agent tries primary search first. If it succeeds, those results are used. If it fails, backup search results are returned instead — the user never sees an error.

Your Task

  • Add fallback logic so backup_search is called automatically when primary_search fails.
  • Ensure the primary search is always attempted first.
  • The fallback should only trigger on actual failures, not on every request.
  • The agent should never expose raw error messages to the user.

Evaluation

Submissions are checked for the following:

  • Primary search is tried first: The agent always attempts the primary tool before falling back.
  • Fallback triggers on failure: When the primary search fails, the backup is automatically called.
  • Agent never crashes: Primary search failures are handled gracefully — the user always gets a result.
  • No unnecessary double calls: The backup is only invoked when the primary actually fails.

Constraints

  • The primary search tool must be tried first
  • Fallback to the secondary search only when the primary fails or returns no results
  • The agent must not call both tools simultaneously on every query
  • Error messages from the primary tool must be caught, not surfaced raw to the user
Starter Code
from langchain_openai import ChatOpenAI
from langchain.agents import AgentExecutor, create_tool_calling_agent
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.tools import tool

llm = ChatOpenAI(model="gpt-4o-mini")

@tool
def primary_search(query: str) -> str:
    """Search using the primary search engine (sometimes fails)."""
    # BUG: This tool intermittently fails and there is no fallback
    import random
    if random.random() < 0.5:
        raise ConnectionError("Primary search service is temporarily unavailable")
    return f"Primary results for '{query}': Article about {query} from TechNews, Research paper on {query} from ArXiv"

@tool
def backup_search(query: str) -> str:
    """Search using the backup search engine (always works)."""
    return f"Backup results for '{query}': Blog post about {query}, Wikipedia article on {query}"

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a research assistant. Search for information to answer questions."),
    ("human", "{input}"),
    ("placeholder", "{agent_scratchpad}"),
])

# BUG: Only primary_search is registered — when it fails, the agent crashes
# TODO: Add fallback logic so backup_search is used when primary fails
agent = create_tool_calling_agent(llm, [primary_search], prompt)
executor = AgentExecutor(agent=agent, tools=[primary_search])

result = executor.invoke({"input": "Find recent research on transformer architectures"})
print(result["output"])
Open in Google Colab
Evaluation Criteria0/4