Measuring change without seeing the system

A call for epistemic humility in intervention evaluation

Abstract

First paragraphs:

In behavioral and applied food system research, intervention studies aimed at improving prac­tices such as healthy eating are often evaluated as if the systems in which they operate are stable and closed. Success is usually measured through spe­cific behavioral outcomes, based on the assump­tion that observed changes can be attributed pri­marily to the intervention itself. However, eating behaviors do not occur in isolation. They are shaped by income, housing conditions, time con­straints, cultural norms, food environments, and policy contexts that extend far beyond any single program. Intervention design already includes assumptions about how the system works, and evaluation frameworks follow those assumptions. Therefore, what evaluation can observe, measure, and interpret is limited before the evaluation even begins.

This problem resonates with recent discussions in food systems scholarship about how a narrow focus on methodological rigor shapes what can be known in complex systems, including JAFSCD’s winter 2025 introduction to a special section of articles on triple rigor (Budowle & Porter, 2025). The introduction highlights the limits of epistemo­logical rigor alone and argues for making space for uncertainty as a condition for more humble and generative knowledge production in complex food systems. This commentary is informed by engage­ment with and review of intervention research in food systems, nutrition, and community development settings. . . .