Back
Rubric Design Studio | EvalCommunity
✦ EvalCommunity Academy

🔍 Rubric Design Studio

Build defensible evaluation rubrics — step by step, with AI as your thinking partner

Step 1 of 5
20%
1

Set the Evaluation Context

Define your program and unpack your broad evaluation question into specific, judgeable dimensions.

💡 A rubric is only as good as the question it answers. Before building criteria, clarify what you are judging and why it matters. AI will help you break down a broad question into specific sub-questions that can become your rubric criteria.
📋 Program Information

Provide basic context about the program or intervention you are evaluating.

❓ Your Broad Evaluation Question

Start with an overarching question. AI will unpack it into 5–6 specific sub-questions suitable for rubric development.

Be as specific as you can. Include the programme name, target population, and the main outcome you want to judge.

AI will generate 5–6 evaluative sub-questions
Unpacking your evaluation question…
🤖 AI-Generated Sub-Questions
    ✏️ Edit the text below before proceeding to Step 2
    2

    Generate Draft Rubric Criteria

    Transform sub-questions into criteria with four performance levels: Poor, Fair, Good, Excellent.

    ⚡ AI generates — you refine. The draft criteria below are a structured starting point. You will review, edit, and validate them with stakeholders in Steps 3 and 4.
    📐 Sub-Questions for Rubric

    Review the sub-questions from Step 1. You can edit them here before generating criteria.

    Add, remove, or rephrase any sub-question to better fit your evaluation design.

    Building your draft rubric criteria…
    🤖 Draft Rubric Criteria
    3

    Stakeholder Engagement & Contextualisation

    Validate your draft rubric with stakeholders and adapt it to local context.

    🌍 Rubrics are social constructs. A rubric embeds values about what counts as "good." AI-generated criteria may not reflect community priorities or cultural context. Use this step to generate discussion questions and adapt your rubric accordingly.
    👥 Generate Stakeholder Discussion Questions

    Paste your draft criteria below, then select your stakeholder audience. AI will generate discussion questions to validate the rubric.

    Generating stakeholder discussion questions…
    🤖 Stakeholder Discussion Questions
    ✍️ Record Stakeholder Feedback

    After your stakeholder sessions, note key feedback here. Use this to refine your rubric before calibration.

    Synthesising stakeholder feedback…
    🤖 Feedback Synthesis & Suggested Revisions
    4

    Test for Consistency & Calibrate

    Identify ambiguous language, sharpen performance level descriptions, and improve inter-rater reliability.

    ⚖️ Calibration is the process of ensuring different evaluators apply the same rubric consistently. AI helps you find ambiguous phrases before applying the rubric to real evidence.
    🔎 Ambiguity Check

    Paste your refined rubric criteria below. AI will flag language that different evaluators might interpret differently and suggest concrete revisions.

    Analysing rubric for ambiguous language…
    🤖 Ambiguity Flags & Suggested Revisions
    📝 Calibration Exercise

    Paste a brief sample of evidence below. Ask AI to simulate how it would rate this evidence against your rubric — helping you test consistency.

    Simulating rating against rubric…
    🤖 Simulated Rating & Reasoning
    5

    Document & Justify Your Judgments

    Draft transparent justifications for each rating and generate your final rubric summary.

    ✅ Transparency builds credibility. Stakeholders trust evaluations when they can follow the reasoning from evidence to judgment. Use AI to draft clear justifications — then review and add your professional judgment.
    ⚖️ Draft a Judgment Justification

    Provide the criterion, evidence, rating, and rubric description. AI will draft a clear justification that links evidence to the rating.

    Drafting justification…
    🤖 Draft Justification
    🔒 Transparency note template: "AI was used to generate an initial draft justification for this criterion based on evidence summaries and rubric descriptions. All final judgments, ratings, and contextual interpretations were made by the evaluation team."
    📄 Generate Final Rubric Summary

    Provide a summary of all your criteria, ratings, and justifications. AI will generate a formatted final rubric narrative suitable for an evaluation report.

    Generating rubric narrative…
    🤖 Final Rubric Narrative

    📥 Export Your Rubric

    Copy the content from any step above into your preferred format. Use the button below to print or save this page as a PDF.

    📌 Ethical Reminder: Rubrics developed with AI assistance should always be validated with stakeholders to ensure they reflect community values and contextual realities. Document the role AI played, and make clear that final evaluative judgments rest with qualified human evaluators. For guidance on ethical AI use in evaluation, see EvalCommunity's AI in M&E resources.