What You Will Build
A practical AI agent that automates evaluation tasks like indicator tracking, report drafting, and data quality review. This tutorial guides you through every step from choosing tools to deploying your first working agent.

Before you start, ensure you have:

• A computer with internet connection (Windows, Mac, or Linux)
• Basic understanding of your M&E workflows (no coding required)
• An account with Anthropic Claude, OpenAI, or Google Gemini (we will walk through this)
• Google account for integration (optional but helpful)
• 30-45 minutes of focused time for setup

Recommendation for Evaluators: Start with Claude or Gemini. Both offer excellent reasoning capabilities and are user-friendly for beginners.

Instructions:

1. Go to claude.ai in your web browser
2. Click the “Sign Up” or “Try Claude” button
3. Enter your email address and create a password
4. Verify your email address (check your inbox for confirmation link)
5. Choose a subscription plan (free tier available for basic use)
6. Complete your profile with your name and organization (optional)

Instructions:

1. Go to gemini.google.com in your browser
2. Sign in with your existing Google account, or create one
3. Accept the terms of service
4. Your Gemini account is now ready to use

Success Check: You should be able to log in and see the chat interface. Try typing a simple test question like “Hello, can you help me with evaluation work?”

For Windows:

1. Go to python.org/downloads
2. Click the yellow “Download Python” button
3. Run the installer when downloaded
4. Check the box “Add Python to PATH” (this is very important)
5. Click “Install Now” and wait for installation to complete

For Mac:

1. Open Terminal (search for it in Spotlight or Applications/Utilities)
2. Type: python3 --version to check if installed
3. If not installed, go to python.org/downloads and download for Mac
4. Follow the installation wizard

1. Go to code.visualstudio.com
2. Click the download button for your operating system
3. Run the installer and follow the prompts
4. Once installed, open VS Code
5. Go to Extensions (icon on left sidebar or Ctrl+Shift+X)
6. Search for and install “Python” extension by Microsoft

On Windows:

1. Open File Explorer
2. Navigate to Documents or Desktop
3. Right-click and select “New Folder”
4. Name it: eval_agent
5. Right-click inside the folder and select “Open with Code” (if VS Code installed)

On Mac:

1. Open Terminal
2. Type: cd ~/Documents
3. Type: mkdir eval_agent
4. Type: cd eval_agent
5. Type: code . to open in VS Code

1. Log in to console.anthropic.com
2. Go to “API Keys” in the left sidebar
3. Click “Create API Key”
4. Give it a name like “eval_agent”
5. Copy the key that appears (you will not see it again)
6. Paste it into your .env file

1. Go to makersuite.google.com/app/apikey
2. Click “Create API Key”
3. Select the project or create a new one
4. Copy the generated API key
5. Paste it into your .env file

Security Tip: Never share your .env file publicly. Add .env to your .gitignore if using version control.

import os
from dotenv import load_dotenv
import anthropic
# Load environment variables
load_dotenv()
# Initialize the Claude client
client = anthropic.Anthropic(
    api_key=os.getenv("ANTHROPIC_API_KEY")
)
def evaluate_indicators(indicator_data):
    """
    AI agent function to analyze M&E indicators
    """
    prompt = f"""
    You are an M&E specialist. Analyze the following indicator data:
    
    {indicator_data}
    
    Please provide:
    1. Summary of overall performance
    2. Indicators that are off-track (below 80% achievement)
    3. Three recommendations for improvement
    """
    
    response = client.messages.create(
        model="claude-3-sonnet-20240229",
        max_tokens=1000,
        temperature=0.3,
        messages=[
            {"role": "user", "content": prompt}
        ]
    )
    
    return response.content[0].text
# Test the agent
if __name__ == "__main__":
    sample_data = """
    Indicator 1: Students enrolled - Target: 1000, Actual: 850 (85%)
    Indicator 2: Teachers trained - Target: 50, Actual: 35 (70%)
    Indicator 3: Schools reached - Target: 20, Actual: 18 (90%)
    """
    
    print("Running evaluation agent...")
    result = evaluate_indicators(sample_data)
    print("\nAI Agent Analysis:")
    print("-" * 40)
    print(result)

Success Check: This code is ready to run! The next step will show you how to execute it.

To run your agent:

1. In VS Code, make sure you are in the eval_agent folder
2. Open Terminal (View → Terminal)
3. Type: python agent.py
4. Press Enter and wait for the response

# Expected output:
Running evaluation agent...
AI Agent Analysis:
----------------------------------------
Based on the indicator data provided, here is my analysis:
1. Overall Performance Summary:
   - Average achievement across indicators: 81.6%
   - 2 out of 3 indicators are on track (>80%)
   - 1 indicator is below target
2. Off-Track Indicators:
   - Teachers trained: 70% achievement (35/50)
     * This is a critical indicator for program success
     * 30% shortfall may impact quality of implementation
3. Recommendations for Improvement:
   - Review training methodology and consider refresher sessions
   - Analyze barriers to teacher participation
   - Consider providing incentives or support materials

def custom_eval_agent(indicator_data, donor_requirements):
    """
    Custom agent tailored to your M&E workflow
    """
    
    # 1. Load your specific data
    # 2. Apply validation rules
    # 3. Generate donor-ready reports
    # 4. Flag risks and issues
    
    prompt = f"""
    You are a senior M&E expert. Analyze this data:
    {indicator_data}
    
    Donor requirements: {donor_requirements}
    
    Generate:
    1. A concise executive summary
    2. Performance table with green/yellow/red status
    3. Key recommendations (max 5)
    4. Evidence gaps that need attention
    """
    
    # Your code here
    pass

You have successfully completed:

• Choosing an AI tool (Claude, Gemini, or OpenAI)
• Creating your account and getting an API key
• Installing Python and required software
• Setting up your project folder and files
• Writing your first agent code
• Running your agent with sample data

Take Your AI Agent Skills to the Next Level

Build professional AI agents for evaluation with the AI Agents for Evaluators Certificate – a practical course designed for M&E professionals.

Course Features: 32 lectures · Lifetime access · Certificate included · Self-paced · 241 students enrolled

Enroll Now – $249 Lifetime Access

Best Value: Bundle with AI in M&E course and save 30%

Ready to go deeper?

Get certified in building AI agents for evaluation with the AI Agents for Evaluators Certificate.

Explore the Full Course