Interactive Quiz – AI in Survey Design

📋 Self-Assessment Quiz

Using AI to Design Better Surveys – 10 Questions

Test your understanding • Learn as you go • No pressure

1 An AI tool suggests rewording a survey question for readability and recommends removing an indicator because it "underperforms in similar contexts." According to the JEDAI framework, which statement correctly applies the framework's principles?

A) Both actions are acceptable as task automation since AI provides efficiency gains B) Rewording is execution (E); removing the indicator crosses into judgment (J) and accountability (A) – human must decide C) The indicator removal is automation because AI identified a pattern; rewording requires human oversight D) Both actions require full human review because AI cannot be trusted with any measurement task

2 You ask AI to generate questions for a "community accountability" indicator. It returns: "Do you know the name of your health facility manager?" "Have you ever complained?" and "Do you feel the facility listens?" Using the SCOPE framework, which evaluation (E) is most accurate?

A) These are well-worded and use appropriate vocabulary – accept them B) They measure awareness and attendance, not accountability (decision-power) – superficial alignment; reject or revise C) They contain leading language that biases responses – rewrite all D) They are too complex for low-literacy respondents – simplify

3 An AI-generated question reads: "Don't you agree that our program has successfully improved community well-being?" Applying the SCAI framework, which two lenses identify the most critical flaws?

A) Clarity and Interpretation – "well-being" is vague B) Social bias and Alignment – it assumes improvement and measures agreement, not well-being C) Alignment and Clarity – it doesn't define what "successfully" means D) Interpretation and Social bias – different groups define well-being differently

4 An AI tool proposes 47 unique survey paths based on combinations of demographics, service usage, and satisfaction. After implementation, completion rates drop from 74% to 38%. Applying the FLOW framework, which diagnosis is most accurate?

A) High Fragmentation (F) and Load (L) created cognitive overload and confusion B) Poor Orientation (O) – respondents didn't know where they were in the survey C) Low Worth (W) – the branches added no analytical value D) Technical issues with skip logic implementation

5 During piloting, AI flags 42% non-response on a question about "household decision-making." Your team immediately wants to reword it. Using the DIAGNOSE framework, what critical step are they skipping?

A) Detect – they haven't confirmed the flag is accurate B) Investigate and Ask – they haven't explored whether non-response varies by subgroup or asked respondents why C) Generate hypotheses – they have only one hypothesis (wording) before narrowing D) Solve – they're designing a solution before identifying the root cause

6 An AI deployment system sends: Day 1 invitation, Day 3 reminder, Day 5 "Last chance – don't miss out," Day 7 automated phone call, Day 10 "We noticed you haven't responded" SMS. Response rates increase 25%. Using the ETHICS framework, which combination of red lines is most clearly crossed?

A) Equity and Transparency B) Harms and Informed Consent C) Coercion risk and Harms D) Stewardship and Equity

7 An AI summary of 500 open-ended responses identifies three main themes, each with 40-60 mentions. Manual review finds 8 responses about "fear of stigma" that don't appear in the summary. According to the VALIDATE framework, what is the correct interpretation and action?

A) AI correctly prioritized frequency; outliers can be footnoted but not featured B) The "Look for missing voices" (L) step reveals that minority perspectives may be programmatically critical – they must be integrated and disclosed C) AI has a bias problem – discard the summary and redo analysis manually D) Ask AI to regenerate with instructions to include all responses, no matter how infrequent

8 A colleague argues: "The JEDAI framework is too restrictive – we should let AI suggest indicators because it has access to global evidence." Using Lesson 1 and 2, what is the most precise counterargument?

A) AI is never accurate enough for indicator selection B) AI can suggest options (SCOPE's "C"), but the "E" (evaluation) and JEDAI's "J" (judgment) must remain human because indicators reflect what matters in context C) Global evidence is irrelevant to local contexts D) Indicator selection should be done by stakeholders only, without AI input

9 An AI tool recommends adding "prefer not to answer" to all questions in a low-literacy context. Using SCOPE and SCAI, what is the appropriate evaluation?

A) Accept – "prefer not to answer" is ethically required for all questions B) Reject entirely – it will confuse and encourage skipping C) Evaluate each question's sensitivity (SCOPE's E) and consider interpretation (SCAI's I) – add only where genuinely needed D) Ask AI to simplify the wording and then accept

10 A team member says: "We used AI to analyze our qualitative data and the summary looks great. Let's put it directly in the report." Which framework-based response is most complete?

A) "AI summaries are never reliable – we should discard it." B) "We need to VALIDATE it: Verify against raw data, Look for missing voices, Interpret nuance, Document AI use, Assess bias, Triangulate, and Explain limitations in the report." C) "Let's run it through another AI tool to verify consistency." D) "Only use it if we paid for the professional version of the AI tool."

Self-Assessment · 7 Lessons · 10 Questions EvalCommunity Academy · No pressure, just learning