1. Why this case matters

Artificial intelligence is increasingly capable of processing large amounts of data, identifying patterns and supporting analysis. But technical performance alone does not determine whether an AI system is useful in a particular community.

Hsu et al. focus on a less explored question: How can scientists co-create AI systems with local communities to address concerns that are specific to a particular place?

Their perspective is based on case studies involving Community Citizen Science (CCS), a framework that brings together community members, scientists and other stakeholders around locally important social or environmental concerns.

The authors describe community empowerment as giving communities greater agency to use technology, data and informed evidence to advocate for social and policy change. This is particularly relevant to evaluators because it shifts attention away from seeing people simply as sources of data and toward seeing them as collaborators in the evidence process.

2. What is Community Citizen Science?

Hsu et al. position Community Citizen Science in relation to community-based participatory research and related approaches including action research, research through design, service design and the PACT framework.

A particularly important distinction concerns the role of community members. In a conventional technology or research project, citizens may be treated primarily as participants or users. In CCS, they are intended to become co-designers and collaborators.

3. Three challenges identified by the authors

Based on their case studies, Hsu et al. identify three major challenges:

1. Co-designing AI systems with local communities — translating complex, local and sometimes informal community needs into implementable AI system features.
2. Collecting and explaining community data using AI — generating, analysing and interpreting community data while dealing with potential biases and the difficulty of explaining AI-generated patterns to non-specialists.
3. Adapting AI systems to long-term social changes — maintaining and changing AI systems as communities, priorities and circumstances evolve.

4. Case study: AI and local air-pollution monitoring

One of the main examples discussed by Hsu et al. involves air-quality monitoring with communities in Pittsburgh.

The projects illustrate how community members can contribute observations and local knowledge while AI and machine-learning techniques help process, connect or interpret those data.

Importantly, the technology was not simply imposed on the community. Community knowledge influenced aspects of system design.

In one example involving computer vision for industrial smoke emissions, characteristics of smoke identified through community knowledge informed the development of the model. Communities also influenced decisions about which areas in videos should be processed.

The article describes another important design decision: the community preferred high precision rather than high recall for the smoke-detection model because the purpose was to identify severe environmental events quickly.

5. Smell Pittsburgh: making local evidence visible

Another case discussed by Hsu et al. is Smell Pittsburgh, where citizens contribute reports about odors and environmental conditions.

The project used machine learning to examine relationships between citizen-contributed odor reports and air-quality sensor measurements.

The important point is not simply that machine learning was used. The authors emphasize the role of an AI pipeline in helping translate complex data relationships into patterns that could become visible to the public.

In this case, pollution patterns became available for public scrutiny and debate. This illustrates a broader challenge for AI-supported evidence systems: analysis is not enough if the people affected by an issue cannot understand or meaningfully interpret the result.

6. RISE: community knowledge becomes part of the dataset

The article also discusses the RISE — Recognizing Industrial Smoke Emissions project.

Citizens affected by air pollution collaborated in annotating videos showing industrial smoke emissions. The resulting dataset was used to train a computer-vision model to recognize industrial smoke.

The authors describe this not only as a social-impact activity but also as a contribution to AI research: community-generated, real-world data can provide a regional dataset for developing and testing AI systems.

This highlights an important principle: data generated with communities can have value beyond the immediate project. The data, collection approach and annotation process can also contribute to future research and model development.

7. Community data can also contain bias

Hsu et al. do not present community-generated data as automatically objective or unbiased.

This is one of the article’s important methodological lessons.

Communities may collect evidence because they already have a strong concern or hypothesis. The authors point out that assumption-driven collection, annotation and analysis can introduce bias. They specifically discuss confirmation bias — the tendency to seek or report information that supports existing beliefs.

In the context of odor reporting, for example, the authors describe the possibility of a higher tendency to report odors associated with pollution events.

8. AI does not remove the need for human interpretation

Another important point in the article concerns the gap between what communities believe an AI system can do and what it actually does.

The authors describe an example in which community members might expect AI to determine automatically whether an industrial site is violating environmental regulations.

However, an AI system may only identify smoke emissions or changes detected through sensors and cameras. Determining whether an event constitutes an actual regulatory violation requires additional human investigation and interpretation.

This distinction is critical for M&E professionals. A model can identify a pattern, classify an observation or estimate a probability without establishing the full causal, legal, programmatic or contextual meaning of that result.

9. The power relationship matters

Hsu et al. place considerable emphasis on the relationship between researchers and communities.

They argue that researcher-centred approaches can create unequal power relationships because researchers may have greater scientific authority and access to resources. Communities can consequently feel that they are being treated as research subjects rather than collaborators.

The authors therefore argue for a shift from designing solutions for people toward working with people affected by the issue.

For M&E, this raises a fundamental question:

If an AI system is introduced into an evaluation without addressing this question, the technology may improve analytical efficiency while leaving the underlying power relationship unchanged.

10. AI systems are not static

Hsu et al. describe the relationship between communities and AI systems as a feedback loop.

Once an AI system becomes part of a community’s activities, it interacts with people and with a changing social environment. Community priorities can change. New evidence can emerge. Users can develop new expectations. The consequences of the technology can also become clearer over time.

The authors therefore argue that AI systems may need continuing adaptation. This could include modifying the machine-learning model, changing the data pipeline, changing how results are interpreted or, under certain conditions, stopping the AI intervention.

Sustaining such systems is not simply a technical problem. The article highlights the need for long-term resources, community engagement, maintenance and funding.

11. Three ways to connect AI research with community needs

One of the most useful sections of the article proposes three approaches for creating both social impact and contributions to research:

• Evaluate social impact as an empirical contribution. Study and document how co-designed AI interventions affect participation, community attitudes, empowerment and local action.
• Curate community data as dataset contributions. Build regional datasets through collaboration with communities, including the processes used to collect and annotate those data.
• Build AI pipelines as methodological contributions. Develop methods that help process, explain and translate AI-generated patterns into information that communities can understand and use.

The authors argue that these approaches can simultaneously support local communities and produce empirical, dataset and methodological contributions to research.

12. What this means for Monitoring & Evaluation

Hsu et al. do not present their work as an M&E methodology. The following application is therefore an EvalCommunity Academy educational adaptation of the article’s ideas for evaluators and development professionals.

The underlying lesson is highly relevant to M&E: when AI becomes part of an evidence system, the quality of the system depends not only on the model but also on the relationship between people, data, technology and decisions.

• Evaluation design: involve affected stakeholders in defining what the AI-supported system is actually intended to accomplish.
• Data collection: examine who generates the data, whose experience is represented and whose experience may be missing.
• Data quality: assess not only technical quality but also incentives, context and possible confirmation bias.
• Analysis: distinguish between what the AI model detects and what an evaluator can legitimately conclude from the evidence.
• Interpretation: make analytical results understandable to the people who need to use them.
• Learning and adaptation: revisit the AI-supported system as programmes, communities and evidence change.

13. An EvalCommunity adaptation: the Community-AI Evidence Cycle

The following cycle is not a framework proposed by Hsu et al. It is an EvalCommunity Academy adaptation designed to translate the article’s lessons into a practical M&E workflow.

14. Practical exercise for M&E professionals

Choose one existing M&E activity in your organization — for example, survey analysis, community feedback, qualitative coding, monitoring dashboards or outcome monitoring.

Then ask:

1. Who currently defines the problem?
2. Who contributes the data?
3. Who decides what counts as useful evidence?
4. Where could AI assist?
5. What could the AI system misunderstand?
6. Could the way data are generated introduce confirmation bias?
7. Can affected stakeholders understand the AI-generated findings?
8. What human judgment remains necessary?
9. What would happen if the programme context changed?
10. Who has the authority to modify or stop the AI-supported process?

15. What could go wrong?

• Researcher dominance: researchers may retain disproportionate control over questions, methods and technology.
• Confirmation bias: data collection and interpretation may unintentionally favour existing assumptions.
• AI capability gap: users may expect the system to answer questions it was never designed to answer.
• Explainability gap: a technically sophisticated model may still produce results that are difficult for communities to interpret.
• Sustainability: a promising system may be difficult to maintain without long-term resources and technical support.
• Context change: a system that works in one phase or setting may require adaptation as community needs change.

16. The deeper lesson for AI in development

The strongest lesson from Hsu et al. is not simply that AI can help communities collect or analyse data.

It is that the way AI is designed and governed can influence whether technology strengthens or weakens community agency.

The authors argue that successful community-based AI requires researchers and designers to work with communities over time, rather than treating community members simply as users or research subjects.

They also emphasize that local contexts are dynamic and hyperlocal. Consequently, a solution developed in one setting should not simply be assumed to work in another.

For M&E and international development, this is a powerful reminder that AI should not replace contextual knowledge. Instead, the goal can be to combine computational capabilities with the knowledge and experience of people who understand the setting directly.

17. Questions for evaluators and development professionals

18. Conclusion

Hsu et al.’s work provides a valuable perspective on a question that becomes increasingly important as AI moves into real-world social and environmental settings:

How do we develop AI with people rather than simply deploy AI for people?

Their case studies show the importance of co-design, community-generated data, explainable analytical pipelines and long-term adaptation. They also demonstrate that community participation introduces its own methodological challenges, including confirmation bias, unequal power relationships, sustainability constraints and gaps between expectations and actual AI capabilities.

For M&E professionals, the lesson is clear: an AI-supported evidence system should be assessed not only by the accuracy or sophistication of its model, but also by who shapes it, whose knowledge it represents, how its outputs are interpreted, and whether it remains useful and accountable in the context where it is used.

Research note and source boundary

This case study is based primarily on Hsu, Y.-C., Huang, T.-H. K., Verma, H., Mauri, A., Nourbakhsh, I., and Bozzon, A. (2022), Empowering local communities using artificial intelligence, published in Patterns, 3(3), 100449.

The original article is a perspective based on case studies of Community Citizen Science and community-based AI. It is not an empirical M&E study and does not claim that the approaches described constitute an M&E methodology.

References to Monitoring & Evaluation, MEAL and international development in this case study are EvalCommunity Academy educational applications of the article’s ideas. They should not be interpreted as findings reported by Hsu et al.

Original research:

Empowering local communities using artificial intelligence

DOI:

10.1016/j.patter.2022.100449

Continue learning with EvalCommunity Academy

Explore more practical resources on the responsible and effective use of AI in Monitoring & Evaluation:

• Use of AI in Evaluation
  
• Principles for AI Use in Monitoring & Evaluation
  
• 50 Ways AI Can Support Monitoring & Evaluation and International Development
  
• AI in Monitoring & Evaluation Certificate