- Ellssel, Pierre
- Küstner,Georg
- Kaczorowska-Dolowy, Magdalena
- Vázquez, Eduardo
- Di Bene, Claudia
- Li, Honghong
- Brizuela Torres, Diego
- Elangovan Vennila, Elansurya
- Vicente Vicente, Jose Luis
- Itzamna Avila-Ortega, Daniel
Enhancing the reliability of literature reviews and evidence synthesis is crucial
for advancing the transformation of agriculture and food (agri-food) systems as
well as for informed decisions and policy making. In this perspective, we argue
that evidence syntheses in the field of agri-food systems research often suffer
from a suite of methodological limitations that substantially increase the risk of
bias, i.e., publication and selection bias, resulting in unreliable and potentially
flawed conclusions and, consequently, poor decisions (e.g., policy direction,
investment, research foci). We assessed 926 articles from the Collaboration for
Environmental Evidence Database of Evidence Reviews (CEEDER) and recent
examples from agri-food systems research to support our reasoning. The
analysis of articles from CEEDER (n = 926) specifically indicates poor quality
(Red) in measures to minimize subjectivity during critical appraisal (98% of all
reviews), application of the eligibility criteria (97%), cross-checking of extracted
data by more than one reviewer (97%), critical appraisal of studies (88%),
establishment of an a priori method/protocol (86%), and transparent reporting
of eligibility decisions (65%). Additionally, deficiencies (Amber) were found in
most articles (>50%) regarding the investigation and discussion of variability
in study findings (89%), comprehensiveness of the search (78%), definition of
eligibility criteria (72%), search approach (64%), reporting of extracted data for
each study (59%), consideration and discussion of the limitations of the synthesis
(56%), documentation of data extraction (54%) and regarding the statistical
approach (52%). To enhance the quality of evidence synthesis in agri-food
science, review authors should use tried-and-tested methodologies and publish
peer-reviewed a priori protocols. Training in evidence synthesis methods should
be scaled, with universities playing a crucial role. It is the shared duty of research
authors, training providers, supervisors, reviewers, and editors to ensure that
rigorous and robust evidence syntheses are made available to decision-makers.
We argue that all these actors should be cognizant of these common mistakes
to avoid publishing unreliable syntheses. Only by thinking as a community can we ensure that reliable evidence is provided to support appropriate decision-making in agri-food systems science., The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. PE received funding through the EU Horizon 2020 Research and Innovation Programme under grant agreement 861924 (SustInAfrica). DIA-O received funding through the Swedish Research Council for Sustainable Development under FORMAS grant number 2020–00371 and the Erling-Persson Family Foundation. DB-T is funded by: the German Centre for Integrative Biodiversity Research (iDiv) – 2021 Flexpool project DeForESG – Deforestation explained by social-ecological dynamics and governance shifts., Peer reviewed