Accuracy with Tools

Example showing an evaluation that runs the provided agent with the provided input and then evaluates the answer that the agent gives.

1. Create a Python file

   accuracy_with_tools.py

   from typing import Optional
   
   from agno.agent import Agent
   from agno.eval.accuracy import AccuracyEval, AccuracyResult
   from agno.models.openai import OpenAIResponses
   from agno.tools.calculator import CalculatorTools
   
   evaluation = AccuracyEval(
       name="Tools Evaluation",
       model=OpenAIResponses(id="gpt-5.2"),
       agent=Agent(
           model=OpenAIResponses(id="gpt-5.2"),
           tools=[CalculatorTools()],
       ),
       input="What is 10!?",
       expected_output="3628800",
   )
   
   result: Optional[AccuracyResult] = evaluation.run(print_results=True)
   assert result is not None and result.avg_score >= 8
   
2. Set up your virtual environment

   Mac

   uv venv --python 3.12
   source .venv/bin/activate
   
3. Install dependencies

   uv pip install -U openai agno
   
4. Export your OpenAI API key

   Mac/Linux

     export OPENAI_API_KEY="your_openai_api_key_here"
   
5. Run Agent

   python accuracy_with_tools.py