Using Generative AI in Qualitative Data Analysis
Review of 192 empirical studies
A Step‑by‑Step Tutorial for Monitoring, Evaluation, and Learning Professionals
Based exclusively on: Generative AI in Qualitative Data Analysis: A Rapid Framework Review (SSRN Working Paper No. 6975579)
About This Tutorial
This tutorial translates evidence from a systematic review of 192 empirical studies (2023–2025) into practical guidance for MEL professionals. The review examined how researchers have applied Generative AI (GenAI) to qualitative data analysis across disciplines, methodologies, and contexts.
Unlike a briefing note that simply summarizes findings, this tutorial provides:
- Step‑by‑step guidance for using GenAI in your qualitative analysis
- Worked examples based on real evaluation scenarios
- Practice tasks with suggested approaches
- Decision frameworks and checklists
- Troubleshooting guidance
Important: This tutorial is based exclusively on the evidence presented in the source paper. All examples, recommendations, and guidance are grounded in what the review actually found — not on speculation or best practices from outside the source material.
Tutorial Learning Objectives
By completing this tutorial, you will be able to:
- Assess whether GenAI is appropriate for your qualitative analysis needs
- Choose an appropriate GenAI tool and implementation approach
- Design effective prompts for coding and thematic analysis
- Distinguish between descriptive and interpretive outputs
- Document your AI use for evaluation reporting
- Identify ethical considerations and plan appropriate safeguards
Before You Begin: What the Evidence Tells Us
The review analyzed 192 studies and found several patterns that should inform your decisions:
| Finding | Implication for Your Practice |
|---|---|
| Most studies used chatbots (62.5%), with GPT models most common (85.2%) | For most evaluators, starting with a commercial chatbot is reasonable |
| Descriptive coding was far more common than interpretive coding | AI is better at organizing than interpreting — plan accordingly |
| Only 10.9% of studies produced pattern‑based themes | Don’t expect AI to deliver deep interpretation automatically |
| Human oversight remained central in 71.9% of studies | You must remain in control of analytical decisions |
| Only 33.9% reported formal ethics approval | You need to be more careful about ethics than many researchers have been |
| Only 2.6% acknowledged environmental impact | Consider the environmental cost of large‑scale AI use |
Tutorial Structure
This tutorial is organized into five modules:
- Module 1: Deciding Whether to Use GenAI
- Module 2: Choosing Your Tools and Implementation Approach
- Module 3: Designing and Running Your Analysis
- Module 4: Documenting and Reporting AI Use
- Module 5: Ethical and Practical Considerations
Each module includes: Key evidence from the review, Step‑by‑step guidance, Worked examples, Practice tasks with answer guides, and Decision checklists.
Module 1: Deciding Whether to Use GenAI
What the Evidence Says
The review found that GenAI applications grew from 13 studies in 2023 to 99 by August 2025, spanning health (31.8%), education (23.4%), and many other fields. However, most studies used GenAI for descriptive coding and topic‑based summaries rather than interpretive analysis. The evidence suggests that GenAI is currently better suited for:
- Organizing and summarizing qualitative information
- Generating descriptive codes
- Producing topic‑based themes
- Working with structured data
GenAI is less well‑suited for:
- Deep interpretive analysis requiring cultural or emotional nuance
- Methods requiring strong reflexivity (e.g., reflexive thematic analysis)
- Analysis where the researcher’s subjectivity is central
Step 1: Conduct a Needs Assessment
1.1 Identify your analysis tasks
| Task Type | Evidence | Recommendation |
|---|---|---|
| Generating descriptive codes | Most common use (45.8%) | Appropriate |
| Creating topic summaries | 62% of studies | Appropriate |
| Identifying interpretive themes | Only 10.9% achieved | Proceed with caution |
| Automating coding | 5.2% automated fully | Not recommended |
| Analyzing non‑English data | Mixed results | Test carefully |
1.2 Assess your data characteristics
The review found that 74.5% of studies used English data. For non‑English data, performance was mixed:
- Encouraging: Turkish, Chinese, German, Italian, Spanish performed reasonably well
- Cautionary: Japanese clinical interviews showed only ~30% agreement on culturally nuanced themes; Swedish idioms were misinterpreted
Decision Rule: If your data involves cultural or emotional nuance, plan for extensive human verification.
Step 2: Define Your Analysis Approach
The review identified three broad qualitative orientations:
| Orientation | Characteristics | GenAI Fit |
|---|---|---|
| Small q (post‑positivist) | Structured coding, reliability, quantification | Strongest fit |
| Medium Q (pragmatic) | Flexible, combines structure and interpretation | Good fit with caution |
| Big Q (interpretive) | Reflexive, subjective, meaning‑focused | Weakest fit |
Practice Task 1.1
Scenario: You are evaluating a community health program in rural Kenya. You have conducted 45 semi‑structured interviews in Swahili and English, exploring participants’ experiences with maternal health services. Your evaluation questions are:
- What barriers do women face in accessing maternal health services?
- How do women describe their experiences with healthcare providers?
- What patterns of meaning emerge around trust, dignity, and respect?
Questions:
- Based on the evidence in the review, which evaluation questions are best suited for GenAI assistance? Why?
- What language‑related concerns should you consider?
- What analytical approach (small q, medium Q, or Big Q) would be most appropriate? Why?
Suggested Answer Guide
Question 1: Question 1 (“what barriers”) is most suited because it involves descriptive coding. Question 2 may be partially suited but requires interpretive attention. Question 3 is least suited because it requires interpretive pattern‑based themes — the review found only 10.9% of studies achieved this.
Question 2: Since data is in both Swahili and English, plan for human verification. The review found that non‑English data with cultural nuance had only ~30% agreement with human researchers.
Question 3: Medium Q would be most appropriate. Small q focuses on reliability, Big Q is not well‑supported by GenAI (only 2 of 18 reflexive TA studies produced pattern‑based themes). Medium Q allows you to use GenAI for structured coding while retaining interpretive control.
Module 2: Choosing Your Tools and Implementation Approach
What the Evidence Says
The review found that researchers accessed GenAI in three ways:
| Implementation | Percentage | Difficulty | When to Use |
|---|---|---|---|
| Chatbot interface | 62.5% | Low | Exploratory, small datasets |
| API‑based apps | 22.9% | High | Large datasets, reproducibility |
| Locally hosted models | 5.7% | High | Sensitive data, privacy |
| GenAI in CAQDAS | 1.6% | Medium | Existing NVivo/MAXQDA users |
Step 1: Select Your Implementation
Decision Flowchart:
→ Data privacy/sensitivity?
→ Use locally hosted model (e.g., Llama 3 locally with Ollama)
→ Large dataset (>500 pages)?
→ Use API‑based implementation (Python + OpenAI API)
→ Small dataset, exploratory?
→ Use chatbot interface (ChatGPT, Claude, Gemini)
→ Already using NVivo/MAXQDA?
→ Consider built‑in AI features (only 1.6% of studies used these)
Step 2: Select Your Model
The review found:
- OpenAI GPT models were used in 85.2% of studies using chatbots
- Google (Gemini), Anthropic (Claude), Meta (Llama), and Mistral each accounted for 7‑8%
Recommendation: Start with GPT‑4 or GPT‑4o (most evidence). If you need privacy, consider locally hosted Llama 3.
Step 3: Set Up Your Workspace
Chatbot (low difficulty):
- Create an account with your chosen provider
- Understand the data policy — researchers were often unclear about data use
- Consider a paid plan for larger volumes
API‑based (high difficulty):
- Obtain an API key from your provider
- Set up a Python environment (or R, using packages like gptworkr)
- Design your pipeline — the review found API use enabled automation, reproducibility, and control over parameters
Step 4: Test Your Setup
The review found that early studies (2023) were mostly proof‑of‑concept exercises. Do the same:
- Take a small sample (2‑3 interviews or ~500 words)
- Run a preliminary analysis
- Compare AI output to your own coding
- Identify issues before scaling up
Practice Task 2.1
Scenario: You have 120 in‑depth interview transcripts (average 45 minutes each, ~8,000 words per transcript) about teacher experiences with a new curriculum in Indonesia. The data includes sensitive information about teacher performance.
Questions:
- Which implementation approach would you choose? Justify your answer.
- What model would you use? Why?
- What are the key technical steps you need to take?
Suggested Answer Guide
Question 1: API‑based implementation. 120 transcripts × ~8,000 words = ~960,000 words — exceeds chat session capacity. The review found API implementation preferred for large datasets because it enables automation, handles volume, and provides reproducibility. For sensitive data, locally hosted would be even better for privacy.
Question 2: GPT‑4 via API is reasonable (85.2% of chatbot studies used GPT models). For data sensitivity, consider locally hosted Llama 3.
Question 3: (1) Obtain API key, (2) Set up Python environment, (3) Write a script to process each transcript, (4) Define your prompt (see Module 3), (5) Run a pilot, (6) Refine prompts, (7) Scale to full dataset, (8) Document your process.
Module 3: Designing and Running Your Analysis
What the Evidence Says
| Aspect | Finding | Implication |
|---|---|---|
| Coding style | 45.8% descriptive noun codes; 5.2% interpretive | Design prompts for interpretive codes if needed |
| Theme style | 62% topic summaries; 10.9% pattern‑based | Actively guide AI toward interpretation |
| Research approaches | 42.7% basic qualitative; named approaches rare | GenAI used with simpler analytical approaches |
| Human‑AI roles | 71.9% human‑led; 13% AI‑led | Keep human oversight central |
Step 1: Design Your Prompt
The review found that prompt design was often under‑documented. Best practice: Document all prompts in full as supplementary material.
1. For descriptive coding (most common use):
You are analyzing interview transcripts from an evaluation of [program name].Your task is to identify and code passages that describe barriers to accessing services.
For each passage you identify:
1. Provide the code as a noun phrase (e.g., “Transportation barriers”, “Cost concerns”)
2. Copy the relevant quote
3. Assign the code
Example:
Code: “Transportation barriers”
Quote: “I have to walk 3 kilometers to the clinic because there’s no bus.”
Transcript to analyze:
[Insert transcript text]
2. For more interpretive coding (less common, requires careful design):
1. Identify the emotion or experience they describe
2. Provide an interpretive code that captures the meaning (not just a noun label)
3. Explain your reasoningExample:
Interpretive code: “Navigating systemic barriers with resilience”
Quote: “Every time I go, they tell me to come back tomorrow. But I keep trying because I have no other option.”
Reasoning: This participant describes a pattern of being turned away but continues to persist. The code captures both the barrier and the response to it.Transcript to analyze:
[Insert transcript text]
3. For theme development:
– A theme should capture a pattern of meaning, not just a topic
– A theme should be expressed as a phrase that conveys meaning
– A theme should be supported by multiple passagesExample:
Topic summary: “Transportation problems”
Pattern-based theme: “Transportation as a barrier to consistent care: Participants described how unreliable transport prevented them from attending appointments regularly, affecting their trust in the health system.”Coded passages:
[Insert coded passages]
Step 2: Run Your Analysis
2.1 Prepare your data
The review found that 47.9% of studies used secondary data, 26.6% used new empirical data, and 17.2% reused existing empirical data.
Recommendation: Always obtain proper ethics clearance before using GenAI with new empirical data. The review found that only 33.9% of studies reported formal ethics approval — you should do better.
2.2 Process your data
Chatbot: Paste text directly, be aware of character limits, upload transcripts as attachments if supported.
API: Write a script that processes each document, use batching if supported, save outputs systematically.
2.3 Review outputs
The review found that most studies (71.9%) had humans in charge. Do the same:
- Check for errors: GenAI models sometimes misread cultural nuances, local idioms, and emotional content
- Check for missing patterns: GenAI may miss subtle interpretive themes
- Check for hallucination: The review found instances of AI producing plausible but incorrect outputs
Step 3: Interpret and Synthesize
3.1 Identify theme style
| Theme Style | Description | Review Finding |
|---|---|---|
| Topic summary | Describes what is in the data | 62% of studies |
| Pattern-based | Captures patterns of shared meaning | 10.9% of studies |
Recommendation: If you need interpretive themes, reinterpret AI‑generated topic summaries yourself. The review found that interpretive codes did not necessarily translate into interpretive themes — you must guide this process.
3.2 Use multiple passes
The review found that studies using API‑based implementations often made multiple passes:
- First pass: Automated coding of all data
- Second pass: Human review and correction
- Third pass: AI refinement based on human feedback
Practice Task 3.1
Scenario: You have three interview excerpts from a program evaluation. Participants were asked: “What has been your experience with the job training program?”
Excerpt 1: “I couldn’t get to the training center because the buses don’t run early enough. I missed three sessions because of it.”
Excerpt 2: “The instructor really took time to explain things. She made sure I understood before moving on. I appreciated that.”
Excerpt 3: “I don’t know if I’ll get a job after this. They said they’d help with placement but I haven’t heard anything.”
Task:
- Write a prompt for descriptive coding.
- Write a prompt for interpretive coding.
- Which type of prompt is more likely to produce reliable results? Why?
- For each excerpt, generate: a descriptive code, an interpretive code, a topic‑summary theme, and a pattern‑based theme.
Suggested Answer Guide
Question 1 (Descriptive): See prompt template above for descriptive coding.
Question 2 (Interpretive): See prompt template above for interpretive coding.
Question 3: The descriptive prompt is more likely to produce reliable results. The review found that 45.8% of studies used descriptive noun codes, while only 5.2% used interpretive codes. GenAI is better at organizing information than at interpreting meaning.
Question 4 (Suggested):
| Excerpt | Descriptive Code | Interpretive Code | Topic‑Summary Theme | Pattern‑Based Theme |
|---|---|---|---|---|
| 1 | “Transportation access barriers” | “Participation constrained by infrastructure gaps” | “Transportation challenges” | “Unequal access: How transport infrastructure shapes participation” |
| 2 | “Positive instructor experience” | “Pedagogical quality as a driver of engagement” | “Instructor quality” | “Quality instruction as a foundation for trust and engagement” |
| 3 | “Job placement uncertainty” | “Unmet expectations undermining confidence” | “Job placement concerns” | “The gap between program promises and post‑program outcomes” |
Module 4: Documenting and Reporting AI Use
What the Evidence Says
The review found significant gaps in documentation:
| Documentation Type | Review Finding | Your Requirement |
|---|---|---|
| Ethics approval | Only 33.9% reported it | Report your approval |
| Positionality/reflexivity | Often absent | State your positionality |
| Prompts | Often not fully disclosed | Disclose all prompts |
| Environmental impact | Only 2.6% acknowledged | Consider and document |
Step 1: Document Your Process
1.1 Data preparation: What data you used, how you prepared it, any anonymization.
1.2 AI tools and settings: Model(s), implementation method, parameters (temperature), date of analysis.
1.3 Prompts used: Provide complete prompt set (not just examples). Consider making prompts available via OSF or GitHub.
1.4 Human‑AI roles: The review found 71.9% of studies were human‑led. Document who steered, interpreted, and decided.
1.5 Quality assurance strategies: Use multiple strategies — independent coding, AI vs human comparison, intercoder reliability, methodological documentation, reflexive journaling, peer insight.
Recommendation: Use multiple quality strategies. Do not rely solely on AI‑human agreement metrics.
Step 2: Report Your Findings
2.1 What to report: Transparency (how AI was used), limitations (descriptive vs interpretive), ethics, reflexivity, environmental impact.
2.2 How to report themes:
Topic‑summary example: “Four themes were identified: Communication, Collaboration, Resource Availability, and Staff Training.”
Pattern‑based example (better practice): “Participants described ‘Navigating systemic barriers through resilience,’ a theme that captures how individuals persistently sought care despite institutional obstacles.”
Practice Task 4.1
Scenario: You have completed a qualitative evaluation using GenAI‑assisted analysis. You need to write the methods section of your report.
Task: Draft the AI section of your methods section. Include: AI tool(s), implementation, prompts (summarize), human researcher role, quality assurance, limitations.
Suggested Answer Guide
Methods: AI‑Assisted Analysis
AI tools and implementation: We used GPT‑4 via the OpenAI API. API access was selected because our dataset consisted of 45 interview transcripts (~360,000 words), exceeding chat interface capacity. The temperature parameter was set to 0.3 to minimize randomness.
Prompts and analysis process: Complete prompts are provided in Appendix A. The analysis proceeded in three stages: (1) Descriptive coding, (2) Review and refinement by two researchers, (3) Theme development through iterative interpretation of coded data.
Human‑AI roles: Researchers remained in control at all stages. AI performed supporting tasks (initial coding and summarization). Researchers steered analysis, interpreted findings, resolved disagreements, and made final decisions.
Quality assurance: Independent review of AI‑generated codes by two researchers; comparison with a human‑coded subset (10%); audit trail of decisions; peer review of theme development; inclusion of participant quotes.
Limitations: AI‑assisted coding is stronger at descriptive organization than interpretive meaning‑making. Themes were developed through human interpretation of AI‑generated codes.
Module 5: Ethical and Practical Considerations
What the Evidence Says
The review found concerning ethical patterns:
- Ethics approval: Only 33.9% reported formal approval; 52.1% said not required; 14.1% gave no ethics information
- Data handling: Many studies used secondary data to avoid consent issues
- Environmental impact: Only 2.6% acknowledged it
- Consent gaps: Some researchers noted that consent forms didn’t cover AI use
Step 1: Plan Your Ethical Approach
1.1 Determine if ethics approval is needed
The review found: New empirical data: 66.7% reported ethics approval; Existing empirical: 60.6%; Secondary: only 10.9%; Mixed: 61.5% reported no ethics information.
Recommendation: If collecting new data, obtain ethics approval. If using secondary data, confirm your use is covered. If data includes personal information, treat it as confidential.
1.2 Address consent issues
The review found that Prescott et al. (2024) noted that participant‑generated text messages would violate confidentiality agreements because consent forms did not specify upload to third‑party vendors.
Recommendation: Include AI use in consent forms; specify data processing and storage; if using commercial AI, note data will be transmitted to third‑party servers; anonymize data before uploading.
1.3 Consider data privacy
The review found that researchers increasingly used locally hosted models to maintain privacy. Vossen et al. (2025) ran models locally so data never left their machine. Mathis et al. (2024) used local LLMs to comply with HIPAA.
Recommendation: For sensitive evaluation data, consider locally hosted models. If using commercial AI, de‑identify data first.
1.4 Consider environmental impact
The review found that only 2.6% of studies acknowledged environmental costs. While the review does not quantify impacts, environmental transparency is an area requiring attention.
Recommendation: Consider the scale of your analysis. For very large datasets using high‑complexity implementations, the environmental impact may be significant. Document your considerations.
Step 2: Addressing Common Ethical Challenges
Challenge 1: Data uploaded to third‑party AI
Recommendation: Anonymize before uploading. Consider API access instead of chat interfaces. Consider local models for sensitive data.
Challenge 2: Existing data not collected with AI consent
Recommendation: Acknowledge the gap in your report. The review found that some researchers acknowledged the absence of AI‑specific consent as a limitation.
Challenge 3: Reproducibility concerns
Recommendation: The review noted that chatbots change without warning (Jiang et al., 2025). Use API implementation for reproducibility. Document the date and model version. Use specific model versions (e.g., “GPT‑4‑0613”) rather than “GPT‑4”.
Step 3: Troubleshooting Common Issues
Issue 1: AI produces topic summaries when you wanted interpretation
Solution: Re‑prompt with specific interpretive instructions; manually reinterpret AI‑generated codes; use human analysis for final theme development.
Issue 2: Low agreement on non‑English data
Solution: The review found non‑English data can have low agreement (e.g., Japanese: ~30% for culturally nuanced themes). Increase human verification.
Issue 3: AI misreads local idioms or cultural context
Solution: Have a native speaker review outputs for cultural accuracy.
Issue 4: AI outputs seem plausible but incorrect
Solution: Always verify key findings against original data. Use multiple quality assurance strategies.
Key Takeaways and Decision Checklist
Summary of What We Know from the Review
| Domain | Key Finding | Implication |
|---|---|---|
| Growth | 13 studies (2023) → 99 (2025) | AI is becoming mainstream |
| Implementation | Chatbots most common; API use rising | Choose based on your needs |
| Data | Mostly secondary; new data rising | Plan ethics accordingly |
| Analysis | Descriptive coding dominates | Manage expectations |
| Themes | Mostly topic summaries | Guide AI toward interpretation |
| Human role | Mostly human‑led | Stay in control |
| Ethics | Inconsistent reporting | Be more careful |
| Inequality | Global North and male dominance | Reflect on access and representation |
Pre‑Analysis Checklist
- I have determined whether GenAI is appropriate for my analysis tasks
- I have selected an implementation approach appropriate to my data volume and sensitivity
- I have obtained ethics approval (for new data) or confirmed secondary data use is covered
- I have updated consent forms to cover AI use (if collecting new data)
- I have planned for data privacy (de‑identification, local hosting if needed)
- I have designed prompts appropriate to my analytical goals
- I have planned for human oversight at all stages
- I have identified quality assurance strategies
- I have planned how I will document AI use
- I have considered environmental impact
During‑Analysis Checklist
- Run pilot on small subset before scaling
- Review AI outputs for errors (cultural nuance, idioms, hallucination)
- Document all prompts used
- Document model and version used
- Document parameters (temperature, etc.)
- Maintain an audit trail of decisions
- Compare AI codes to human codes on a sample
- Re‑prompt or refine if outputs don’t meet needs
- Keep human researchers in control of interpretation
Post‑Analysis Checklist
- Report AI use in methods section
- Include complete prompts in supplementary materials
- State limitations of AI use
- Acknowledge any ethics or consent gaps
- Consider environmental impact disclosure
- Include participant quotes to support themes
- Distinguish between AI‑generated and human‑generated findings
Final Reflection
The review concludes that the field of GenAI in qualitative analysis has “expanded faster than its methodological foundation has matured.” For MEL professionals, this means:
- You can benefit from GenAI — it is effective for organizing, coding, and summarizing qualitative data.
- You must remain cautious — AI is not yet capable of deep interpretive analysis.
- You must be transparent — document your use, your prompts, and your limitations.
- You must be ethical — obtain appropriate approvals, protect data, and consider environmental impact.
- You must stay in control — AI is a tool, not a replacement for evaluation expertise.
Reference:
Nguyen‑Xuan, K., Nguyen, T.‑A., & Nguyen‑Trung, K. (2026). Generative AI in Qualitative Data Analysis: A Rapid Framework Review (SSRN Working Paper No. 6975579).
All guidance in this tutorial is based exclusively on the evidence presented in the source paper. Specific findings referenced are drawn from the review’s ten key findings (pages 9‑29).
