✅ What worked (superficially)

• The writing sounded very good — professional, fluent, and evaluation-appropriate
• High-level messages and insights were appropriate for the topic
• The structure followed expected evaluation synthesis conventions
• To a non-expert reader, the output would have appeared credible

❌ What failed (catastrophically)

• The model fabricated evidence and examples that did not exist in the source reports
• It invented examples of interventions that never took place
• It invented text that did not exist in any of the six PPARs
• When called out on these fabrications, the chatbot denied it — insisting the examples were in the reports when they were not

❌ Mistake #1: Asking for Synthesis Without Component Validation

What they did: Asked the model to directly synthesize six full reports in one go, without first validating that the model could accurately extract, summarize, and represent information from each individual report.

✅ What they should have done: Follow the modular approach from the 2025 guidance note: (1) extract key information from each report, (2) validate extraction faithfulness against source, (3) summarize each report individually, (4) validate summaries, (5) then synthesize validated summaries. “It is important to unpack the various analytical steps.”

❌ Mistake #2: No Ground Truth or Validation Metrics

What they did: Assessed the synthesis qualitatively (“sounded good”) without establishing systematic metrics for faithfulness, relevance, or coherence. By the time they discovered fabrications, the synthesis was already complete.

✅ What they should have done: Establish metrics BEFORE generating the synthesis. The 2025 guidance note uses faithfulness (0/1), relevance (1-5 Likert), and coherence (1-5 Likert). Validate each component against human-annotated ground truth before moving to synthesis.

❌ Mistake #3: No Citation or Source Attribution Requirement

What they did: Allowed the model to generate claims without requiring explicit citations to specific source documents, sentences, or page numbers.

✅ What they should have done: The 2025 guidance note advises: “Asking the model to include references to the source document(s) in its response can help with prompt refinement… be clear that it should cite the specific summaries in its response.” Require paragraph-level or sentence-level citations for every claim.

❌ Mistake #4: No “Unknown” Category or Confidence Signaling

What they did: The model was forced to produce an answer regardless of whether the evidence existed, leading to fabrication when information was absent.

✅ What they should have done: The 2025 guidance note states: “Provide ‘unknown’ or ‘not applicable’ as a category. A limitation of closed LLMs is that they are configured to always generate some response… This implies the model may generate speculative results when it encounters insufficient or low-quality instructions or input data.” Allow the model to acknowledge absence of evidence.

❌ Mistake #5: No Human Validation Before Accepting Output

What they did: Treated the iterative rounds of prompting as sufficient quality control, without systematic human verification of each claim against source documents.

✅ What they should have done: The 2025 guidance note is explicit: “The manual review component is mandatory in our workflows when LLM or machine learning are used.” This means line-by-line verification of every synthesized claim against source documents—not just a final read-through.

📋 2023 Approach (Failed)

• Synthesize 6 full PPARs directly
• Multiple iterative prompts
• No component validation
• No citation requirements
• Result: Fabricated evidence

✅ 2024 Approach (Successful)

• Synthesize 6 summaries (200 words each), not full reports
• Single 500-word synthesis
• Each summary already validated for faithfulness
• Model required to cite specific summaries
• Result: Faithfulness 1.00, Coherence 5.00

✅ Successful experiments (6)

• Writing code for preprocessing textual data
• Explaining programming code
• Simple classification (disaster risk reduction)
• Sentiment analysis
• Econometric analysis (R code generation)
• Summarizing individual documents

❌ Unsuccessful experiments (3)

• Generating synthetic images for data augmentation
• Conducting a literature review (SLR) — Case Study 1
• Conducting an evaluation synthesis — This case study

These 3 failures became the foundation for the 2025 guidance note.

🔗 Never skip component validation

Synthesize validated summaries, not raw documents. The 2025 guidance note emphasizes: “Because the synthesis was conducted with summaries of the source documents, the results were likely better.” Validate extraction and summarization before synthesis.

📎 Require citations for every claim

The 2024 synthesis succeeded partly because the model was required to reference specific summaries. Always ask the model to cite its sources at the sentence or paragraph level. Without citations, you cannot verify faithfulness.

🤷 Provide an “unknown” category

LLMs are forced to respond. Give them permission to say “not found” or “insufficient information.” This prevents the model from fabricating evidence when information is absent—the core failure of the 2023 synthesis.

👁️ Mandatory human verification

The guidance note is unambiguous: “The manual review component is mandatory.” For synthesis, this means line-by-line verification of every claim against source documents. Do not trust the model’s self-assessment—it will deny fabricating evidence.

Why did the model fabricate evidence when it had the source documents?

LLMs are not databases or retrieval systems. They are next-token predictors. When asked to provide “specific examples” from six documents, the model may prioritize fluency and apparent completeness over accuracy. Without citation requirements and with no “unknown” option, the model fills gaps with plausible-sounding fabrications. This is why the 2025 guidance note emphasizes: “LLMs do not always generate aligned, authoritative, or accurate responses… their responses must be validated before use in our work.”

Why did the chatbot deny fabricating evidence when confronted?

LLMs have no internal mechanism to distinguish between information from source documents and information generated from their training data. They also lack a “confidence” signal. When asked “Is this example in the report?” the model will generate a response based on what seems most plausible—not based on actual retrieval. The 2025 guidance note warns: “Due to the opaque nature of LLMs’ inner workings, it is not possible for humans to interpret their ‘decision-making process’… This challenge can be mitigated to some extent by including instructions in the prompt for the model to justify its reasoning.” However, even justifications can be fabricated.

Can I use LLMs for evaluation synthesis today?

Yes—but only with strict protocols. Based on the 2025 guidance note: (1) Break synthesis into extraction → summarization → synthesis steps. (2) Validate each component against human-annotated ground truth. (3) Require citations for every claim. (4) Provide an “unknown” category. (5) Conduct mandatory human verification of all synthesized claims against source documents. Under these conditions, the 2024 experiments achieved perfect faithfulness (1.00) and coherence (5.00).

What metrics should I use to evaluate synthesis quality?

The 2025 guidance note recommends three criteria for generative tasks: Faithfulness (0/1 scale — is information factually consistent with source?), Relevance (1-5 Likert — is selected content the most important?), and Coherence (1-5 Likert — is the response logically structured?). For synthesis specifically, also track citation accuracy and calculate the percentage of claims that can be directly traced to source documents.