Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 41
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Adherence to reporting guidelines increases the number of citations: the argument for including a methodologist in the editorial process and peer-review

  • Vilaró, Marta
  • Cortés, Jordi
  • Selva-O'Callaghan, Albert
  • Urrutia, Agustín
  • Ribera, Josep Maria
  • Cardellach, Francesc
  • Basagaña Flores, Xavier
  • Elmore, Matthew
  • Vilardell, Miquel
  • Altman, Douglas
  • González, José-Antonio
  • Cobo Valeri, Erik
Background: From 2005 to 2010, we conducted 2 randomized studies on a journal (Medicina Clínica), where we took manuscripts received for publication and randomly assigned them to either the standard editorial process or to additional processes. Both studies were based on the use of methodological reviewers and reporting guidelines (RG). Those interventions slightly improved the items reported on the Manuscript Quality Assessment Instrument (MQAI), which assesses the quality of the research report. However, masked evaluators were able to guess the allocated group in 62% (56/90) of the papers, thus presenting a risk of detection bias. In this post-hoc study, we analyse whether those interventions that were originally designed for improving the completeness of manuscript reporting may have had an effect on the number of citations, which is the measured outcome that we used. Methods: Masked to the intervention group, one of us used the Web of Science (WoS) to quantify the number of citations that the participating manuscripts received up December 2016. We calculated the mean citation ratio between intervention arms and then quantified the uncertainty of it by means of the Jackknife method, which avoids assumptions about the distribution shape. Results: Our study included 191 articles (99 and 92, respectively) from the two previous studies, which all together received 1336 citations. In both studies, the groups subjected to additional processes showed higher averages, standard deviations and annual rates. The intervention effect was similar in both studies, with a combined estimate of a 43% (95% CI: 3 to 98%) increase in the number of citations. Conclusions: We interpret that those effects are driven mainly by introducing into the editorial process a senior methodologist to find missing RG items. Those results are promising, but not definitive due to the exploratory nature of the study and some important caveats such as: the limitations of using the number of citations as a measure of scientific impact; and the fact that our study is based on a single journal. We invite journals to perform their own studies to ascertain whether or not scientific repercussion is increased by adhering to reporting guidelines and further involving statisticians in the editorial process., DGA, JAG and EC belong to MiRoR (Methods in Research on Research), which has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 676207. EC and JC are partially supported by the grants MTM2015–64465-C2–1-R (MINECO/FEDER, UE) from the Secretaría de Estado de Investigación, Desarrollo e Innovación del Ministerio de Economía y Competitividad (Spain) and through grant 2017 SGR 622 from the Departament d’Economia i Coneixement de la Generalitat de Catalunya (Spain). The funding bodies had no role in the study design, data collection and analysis, results interpretation, or in writing the manuscript.
Dipòsit Digital de la UB
Publication Artículo científico (JournalArticle)

Adherence to reporting guidelines increases the number of citations: the argument for including a methodologist in the editorial process and peer-review

  • Vilaró, Marta
  • Cortés, Jordi
  • Selva O'Callaghan, Albert
  • Urrutia, Agustín
  • Ribera, Josep Maria
  • Cardellach, Francesc
  • Basagaña, Xavier
  • Elmore, Matthew
  • Vilardell, Miquel
  • Altman, Douglas G.
  • González, José Antonio
  • Cobo, Erik
Background: From 2005 to 2010, we conducted 2 randomized studies on a journal (Medicina Clínica), where we took manuscripts received for publication and randomly assigned them to either the standard editorial process or to additional processes. Both studies were based on the use of methodological reviewers and reporting guidelines (RG). Those interventions slightly improved the items reported on the Manuscript Quality Assessment Instrument (MQAI), which assesses the quality of the research report. However, masked evaluators were able to guess the allocated group in 62% (56/90) of the papers, thus presenting a risk of detection bias. In this post-hoc study, we analyse whether those interventions that were originally designed for improving the completeness of manuscript reporting may have had an effect on the number of citations, which is the measured outcome that we used. Methods: Masked to the intervention group, one of us used the Web of Science (WoS) to quantify the number of citations that the participating manuscripts received up December 2016. We calculated the mean citation ratio between intervention arms and then quantified the uncertainty of it by means of the Jackknife method, which avoids assumptions about the distribution shape. Results: Our study included 191 articles (99 and 92, respectively) from the two previous studies, which all together received 1336 citations. In both studies, the groups subjected to additional processes showed higher averages, standard deviations and annual rates. The intervention effect was similar in both studies, with a combined estimate of a 43% (95% CI: 3 to 98%) increase in the number of citations. Conclusions: We interpret that those effects are driven mainly by introducing into the editorial process a senior methodologist to find missing RG items. Those results are promising, but not definitive due to the exploratory nature of the study and some important caveats such as: the limitations of using the number of citations as a measure of scientific impact; and the fact that our study is based on a single journal. We invite journals to perform their own studies to ascertain whether or not scientific repercussion is increased by adhering to reporting guidelines and further involving statisticians in the editorial process.
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Using the STROBE statement: survey findings emphasized the role of journals in enforcing reporting guidelines

  • Sharp, Melissa K.
  • Bertizzolo, Lorenzo
  • Rius Carrasco, Roser|||0000-0002-6458-635X
  • Wager, Elisabeth
  • Gómez Melis, Guadalupe|||0000-0003-4252-4884
  • Hren, Darko
© 2019 Elsevier. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/, Objectives The objective of the study was to identify factors affecting the use of the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement, specifically authors’ attitudes toward and experiences with it. Study Design and Setting An online survey was distributed to authors of observational studies recruited via social media, personal network snowballing, and mass mailings using targeted search strategies. Data on demographics, awareness, motivators, and usage were collected in conjunction with a modified Unified Theory of Acceptance and Use of Technology (UTAUT) scale on which confirmatory factor analysis (CFA) was performed. Results One thousand fifteen participants completed the survey. Of these, 185 (18.2%) indicated they had never heard of STROBE nor used it previously, 195 (19.2%) had heard of it but never used it, and 635 (62.6%) had used it. Journals promoting STROBE were both key motivators and awareness mechanisms; peers and educational workshops were also important influencing factors to a lesser degree. The internal consistency of the modified UTAUT scale was strong (Cronbach's alpha = 0.94). CFA supported a four-factor model with 23 questions. Conclusion The endorsement of STROBE by journals is key to authors’ awareness and use of the guideline. We tested and validated our scale which can guide future research on reporting guidelines., The authors would like to acknowledge the Methods in Research on Research (MiRoR) network for their support and guidance. The authors would also like to thank Dr. Michail Kovanis for assistance gathering author emails and Ms. Alice Biggane for her assistance with the recruitment strategy., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Adherence to reporting guidelines increases the number of citations: the argument for including a methodologist in the editorial process and peer-review

  • Vilaró, Marta
  • Cortés Martínez, Jordi|||0000-0002-3764-0795
  • Selva O’Callaghan, Albert
  • Urrutia, Agustí
  • Ribera, Josep Maria
  • Cardellach López, Francesc
  • Basagaña Flores, Xavier
  • Elmore, Matthew
  • Vilardell, Miquel
  • Altman, D.G.
  • González Alastrué, José Antonio|||0000-0002-3113-4779
  • Cobo Valeri, Erik|||0000-0002-3534-5602
From 2005 to 2010, we conducted 2 randomized studies on a journal (Medicina Clínica), where we took manuscripts received for publication and randomly assigned them to either the standard editorial process or to additional processes. Both studies were based on the use of methodological reviewers and reporting guidelines (RG). Those interventions slightly improved the items reported on the Manuscript Quality Assessment Instrument (MQAI), which assesses the quality of the research report. However, masked evaluators were able to guess the allocated group in 62% (56/90) of the papers, thus presenting a risk of detection bias. In this post-hoc study, we analyse whether those interventions that were originally designed for improving the completeness of manuscript reporting may have had an effect on the number of citations, which is the measured outcome that we used., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

A systematic review describes models for recruitment prediction at the design stage of a clinical trial

  • Gkioni, Efstathia
  • Rius Carrasco, Roser|||0000-0002-6458-635X
  • Dodd, Susanna
  • Gamble, Carrol
© 2019 Elsevier. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/, Patient recruitment in clinical trials is challenging with failure to recruit to time and target sample size common. This may be caused by unanticipated problems or by overestimation of the recruitment rate. This study is a systematic review of statistical models to predict recruitment at the design stage of clinical trials, The authors would like to thank Ms Van Nguyen Thu, PhD candidate, on the MiRoR project, and Professor Isabelle Boutron, Paris Descartes University, for their help in reviewing the manuscript before submission. This project is part of the MiRoR (Methods in Research on Research) Joint Doctoral Training Programme. The authors would like to acknowledge the MiRoR network for their support and guidance. The European Union's Horizon 2020 Marie Sklodowska Curie Innovative Training Networks–European Joint Doctorate (ITN-EJD) under grant agreement No. 676207, fund MiRoR., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Tools used to assess the quality of peer review reports: a methodological systematic review

  • Superchi, Cecilia|||0000-0002-5375-6018
  • González Alastrué, José Antonio|||0000-0002-3113-4779
  • Solà, Ivan
  • Cobo Valeri, Erik|||0000-0002-3534-5602
  • Hren, Darko
  • Boutron, Isabelle
Background A strong need exists for a validated tool that clearly defines peer review report quality in biomedical research, as it will allow evaluating interventions aimed at improving the peer review process in well-performed trials. We aim to identify and describe existing tools for assessing the quality of peer review reports in biomedical research. Methods We conducted a methodological systematic review by searching PubMed, EMBASE (via Ovid) and The Cochrane Methodology Register (via The Cochrane Library) as well as Google® for all reports in English describing a tool for assessing the quality of a peer review report in biomedical research. Data extraction was performed in duplicate using a standardized data extraction form. We extracted information on the structure, development and validation of each tool. We also identified quality components across tools using a systematic multi-step approach and we investigated quality domain similarities among tools by performing hierarchical, complete-linkage clustering analysis. Results We identified a total number of 24 tools: 23 scales and 1 checklist. Six tools consisted of a single item and 18 had several items ranging from 4 to 26. None of the tools reported a definition of ‘quality’. Only 1 tool described the scale development and 10 provided measures of validity and reliability. Five tools were used as an outcome in a randomized controlled trial (RCT). Moreover, we classified the quality components of the 18 tools with more than one item into 9 main quality domains and 11 subdomains. The tools contained from two to seven quality domains. Some domains and subdomains were considered in most tools such as the detailed/thorough (11/18) nature of reviewer’s comments. Others were rarely considered, such as whether or not the reviewer made comments on the statistical methods (1/18). Conclusion Several tools are available to assess the quality of peer review reports; however, the development and validation process is questionable and the concepts evaluated by these tools vary widely. The results from this study and from further investigations will inform the development of a new tool for assessing the quality of peer review reports in biomedical research., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Scoping review on interventions to improve adherence to reporting guidelines in health research

  • Blanco de Tena Davila, David|||0000-0003-2961-9328
  • Altman, D.G.
  • Moher, David
  • Boutron, Isabelle
  • Kirkham, Jamie
  • Cobo Valeri, Erik|||0000-0002-3534-5602
Objectives The goal of this study is to identify, analyse and classify interventions to improve adherence to reporting guidelines in order to obtain a wide picture of how the problem of enhancing the completeness of reporting of biomedical literature has been tackled so far. Design Scoping review. Search strategy We searched the MEDLINE, EMBASE and Cochrane Library databases and conducted a grey literature search for (1) studies evaluating interventions to improve adherence to reporting guidelines in health research and (2) other types of references describing interventions that have been performed or suggested but never evaluated. The characteristics and effect of the evaluated interventions were analysed. Moreover, we explored the rationale of the interventions identified and determined the existing gaps in research on the evaluation of interventions to improve adherence to reporting guidelines. Results 109 references containing 31 interventions (11 evaluated) were included. These were grouped into five categories: (1) training on the use of reporting guidelines, (2) improving understanding, (3) encouraging adherence, (4) checking adherence and providing feedback, and (5) involvement of experts. Additionally, we identified lack of evaluated interventions (1) on training on the use of reporting guidelines and improving their understanding, (2) at early stages of research and (3) after the final acceptance of the manuscript. Conclusions This scoping review identified a wide range of strategies to improve adherence to reporting guidelines that can be taken by different stakeholders. Additional research is needed to assess the effectiveness of many of these interventions, Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Statistical models to predict recruitment in clinical trials were rarely used by statisticians in UK and European networks

  • Gkioni, Efstathia
  • Dodd, Susanna
  • Rius Carrasco, Roser|||0000-0002-6458-635X
  • Gamble, Carrol
© 2020. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/, Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Effect of an editorial intervention to improve the completeness of reporting of randomised trials: a randomised controlled trial

  • Blanco de Tena Davila, David|||0000-0003-2961-9328
  • Cobo Valeri, Erik|||0000-0002-3534-5602
Objective To evaluate the impact of an editorial intervention to improve completeness of reporting of reports of randomised trials. Design Randomised controlled trial (RCT). Setting BMJ Open’s quality improvement programme. Participants 24 manuscripts describing RCTs. Interventions We used an R Shiny application to randomise manuscripts (1:1 allocation ratio, blocks of 4) to the intervention (n=12) or control (n=12) group. The intervention was performed by a researcher with expertise in the content of the Consolidated Standards of Reporting Trials (CONSORT) and consisted of an evaluation of completeness of reporting of eight core CONSORT items using the submitted checklist to locate information, and the production of a report containing specific requests for authors based on the reporting issues found, provided alongside the peer review reports. The control group underwent the usual peer review. Outcomes The primary outcome is the number of adequately reported items (0–8 scale) in the revised manuscript after the first round of peer review. The main analysis was intention-to-treat (n=24), and we imputed the scores of lost to follow-up manuscripts (rejected after peer review and not resubmitted). The secondary outcome is the proportion of manuscripts where each item was adequately reported. Two blinded reviewers assessed the outcomes independently and in duplicate and solved disagreements by consensus. We also recorded the amount of time to perform the intervention. Results Manuscripts in the intervention group (mean: 7.01; SD: 1.47) were more completely reported than those in the control group (mean: 5.68; SD: 1.43) (mean difference 1.43, 95% CI 0.31 to 2.58). We observed the main differences in items 6a (outcomes), 9 (allocation concealment mechanism), 11a (blinding) and 17a (outcomes and estimation). The mean time to perform the intervention was 87 (SD 42) min. Conclusions We demonstrated the benefit of involving a reporting guideline expert in the editorial process. Improving the completeness of RCTs is essential to enhance their usability. Trial registration number NCT03751878., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

The STROBE extensions: protocol for a qualitative assessment of content and a survey of endorsement

  • Sharp, Melissa K.
  • Utropia, Ana
  • Gómez Melis, Guadalupe|||0000-0003-4252-4884
  • Cobo Valeri, Erik|||0000-0002-3534-5602
  • Wager, Elisabeth
  • Hren, Darko
Introduction The STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) Statement was developed in response to inadequate reporting of observational studies. In recent years, several extensions to STROBE have been created to provide more nuanced field-specific guidance for authors. The content and the prevalence of extension endorsement have not yet been assessed. Accordingly, there are two aims: (1) to classify changes made in the extensions to identify strengths and weaknesses of the original STROBE checklist and (2) to determine the prevalence and typology of endorsement by journals in fields related to extensions. Methods and analysis Two independent researchers will assess additions in each extension. Additions will be coded as â field specific' (FS) or â not field specific' (NFS). FS is defined as particularly relevant information for a single field and guidance provided generally cannot be extrapolated beyond that field. NFS is defined as information that reflects epidemiological or methodological tenets and can be generalised to most, if not all, types of observational research studies. Intraclass correlation will be calculated to measure reviewers' concordance. On disagreement, consensus will be sought. Individual additions will be grouped by STROBE checklist items to identify the frequency and distribution of changes. Journals in fields related to extensions will be identified through National Library of Medicine PubMed Broad Subject Terms, screened for eligibility and further distilled via Ovid MEDLINE® search strategies for observational studies. Text describing endorsement will be extracted from each journal's website. A classification scheme will be created for endorsement types and the prevalence of endorsement will be estimated. Analyses will use NVivo V.11 and SAS University Edition. Ethics and dissemination This study does not require ethical approval as it does not involve human participants. This study has been preregistered on Open Science Framework., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Are CONSORT checklists submitted by authors adequately reflecting what information is actually reported in published papers?

  • Blanco de Tena Davila, David|||0000-0003-2961-9328
  • Biganne, Alice M.
  • Cobo Valeri, Erik|||0000-0002-3534-5602
Background: Compulsory submission of a checklist from the relevant reporting guideline is one of the most widespread journal requirements aiming to improve completeness of reporting. However, the current suboptimal levels of adherence to reporting guidelines observed in the literature may indicate that this journal policy is not having a significant effect. Findings: We explored whether authors provided the appropriate CONSORT checklist extension for their study and whether there were inconsistencies between what authors claimed on the submitted checklist and what was actually reported in the published paper. We randomly selected 12 randomized trials from three journals that provide the originally submitted checklist and analyzed six core CONSORT items. Only one paper used the appropriate checklist extension and had no inconsistencies between what was claimed in the submitted checklist and what was reported in the published paper. Conclusion: Journals should take further actions to take full advantage of the requirement for the submission of fulfilled CONSORT checklists, thus ensuring that these checklists reflect what is reported in the manuscript., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Interventions to improve adherence to reporting guidelines in health research: a scoping review protocol

  • Blanco de Tena Davila, David|||0000-0003-2961-9328
  • Kirkham, Jamie
  • Altman, D.G.
  • Moher, David
  • Boutron, Isabelle
  • Cobo Valeri, Erik|||0000-0002-3534-5602
There is evidence that the use of some reporting guidelines, such as the Consolidated Standards for Reporting Trials, is associated with improved completeness of reporting in health research. However, the current levels of adherence to reporting guidelines are suboptimal. Over the last few years, several actions aiming to improve compliance with reporting guidelines have been taken and proposed. We will conduct a scoping review of interventions to improve adherence to reporting guidelines in health research that have been evaluated or suggested, in order to inform future interventions., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Taking advantage of unexpected WebCONSORT results

  • Cobo Valeri, Erik|||0000-0002-3534-5602
  • González Alastrué, José Antonio|||0000-0002-3113-4779
To estimate treatment effects, trials are initiated by randomising patients to the interventions under study and finish by comparing patient evolution. In order to improve the trial report, the CONSORT statement provides authors and peer reviewers with a guide of the essential items that would allow research replication. Additionally, WebCONSORT aims to facilitate author reporting by providing the items from the different CONSORT extensions that are relevant to the trial being reported. WebCONSORT has been estimated to improve the proportion of reported items by 0.04 (95% CI, –0.02 to 0.10), interpreted as “no important difference”, in accordance with the scheduled desired scenario of a 0.15 effect size improvement. However, in a non-scheduled analysis, it was found that, despite clear instructions, around a third of manuscripts selected for trials by the editorial staff were not actually randomised trials. We argue that surprises benefit science, and that further research should be conducted in order to improve the performance of editorial staff., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Development of ARCADIA: a tool for assessing the quality of peer-review reports in biomedical research

  • Superchi, Cecilia|||0000-0002-5375-6018
  • Hren, Darko
  • Blanco de Tena Davila, David|||0000-0003-2961-9328
  • Rius Carrasco, Roser|||0000-0002-6458-635X
  • Recchioni, Alessandro
  • Boutron, Isabelle
  • González Alastrué, José Antonio|||0000-0002-3113-4779
Objective To develop a tool to assess the quality of peer- review reports in biomedical research. Methods We conducted an online survey intended for biomedical editors and authors. The survey aimed to (1) determine if participants endorse the proposed definition of peer- review report quality; (2) identify the most important items to include in the final version of the tool and (3) identify any missing items. Participants rated on a 5- point scale whether an item should be included in the tool and they were also invited to comment on the importance and wording of each item. Principal component analysis was performed to examine items redundancy and a general inductive approach was used for qualitative data analysis. Results A total of 446 biomedical editors and authors participated in the survey. Participants were mainly male (65.9%), middle- aged (mean=50.3, SD=13) and with PhD degrees (56.4%). The majority of participants (84%) agreed on the definition of peer- review report quality we proposed. The 20 initial items included in the survey questionnaire were generally highly rated with a mean score ranging from 3.38 (SD=1.13) to 4.60 (SD=0.69) (scale 1–5). Participants suggested 13 items that were not included in the initial list of items. A steering committee composed of five members with different expertise discussed the selection of items to include in the final version of the tool. The final checklist includes 14 items encompassed in five domains (Importance of the study, Robustness of the study methods, Interpretation and discussion of the study results, Reporting and transparency of the manuscript, Characteristics of peer reviewer’s comments). Conclusion Assessment of Review reports with a Checklist Available to eDItors and Authors tool could be used regularly by editors to evaluate the reviewers’ work, and also as an outcome when evaluating interventions to improve the peer- review process., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

A scoping review protocol on the roles and tasks of peer reviewers in the manuscript review process in biomedical journals

  • Glonti, Ketevan
  • Cauchi, Daniel
  • Cobo Valeri, Erik|||0000-0002-3534-5602
  • Boutron, Isabelle
  • Moher, David
  • Hren, Darko
Introduction The primary functions of peer reviewers are poorly defined. Thus far no body of literature has systematically identified the roles and tasks of peer reviewers of biomedical journals. A clear establishment of these can lead to improvements in the peer review process. The purpose of this scoping review is to determine what is known on the roles and tasks of peer reviewers. Methods We will use the methodological framework first proposed by Arksey and O'Malley and subsequently adapted by Levac et al and the Joanna Briggs Institute. The scoping review will include all study designs, as well as editorials, commentaries and grey literature. The following eight electronic databases will be searched (from inception to May 2017): Cochrane Library, Cumulative Index to Nursing and Allied Health Literature, Educational Resources Information Center, EMBASE, MEDLINE, PsycINFO, Scopus and Web of Science. Two reviewers will use inclusion and exclusion criteria based on the 'Population-Concept-Context' framework to independently screen titles and abstracts of articles considered for inclusion. Full-text screening of relevant eligible articles will also be carried out by two reviewers. The search strategy for grey literature will include searching in websites of existing networks, biomedical journal publishers and organisations that offer resources for peer reviewers. In addition we will review journal guidelines to peer reviewers on how to perform the manuscript review. Journals will be selected using the 2016 journal impact factor. We will identify and assess the top five, middle five and lowest-ranking five journals across all medical specialties. Ethics and dissemination This scoping review will undertake a secondary analysis of data already collected and does not require ethical approval. The results will be disseminated through journals and conferences targeting stakeholders involved in peer review in biomedical research., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

National citation patterns of NEJM, The Lancet, JAMA and The BMJ in the lay press: a quantitative content analysis

  • Casino, Gonzalo
  • Rius Carrasco, Roser|||0000-0002-6458-635X
  • Cobo Valeri, Erik|||0000-0002-3534-5602
Objectives: To analyse the total number of newspaper articles citing the four leading general medical journals and to describe national citation patterns. Design: Quantitative content analysis. Setting/sample: Full text of 22 general newspapers in 14 countries over the period 2008–2015, collected from LexisNexis. The 14 countries have been categorised into four regions: the USA, the UK, Western World (European countries other than the UK, and Australia, New Zealand and Canada) and Rest of the World (other countries). Main outcome measure Press citations of four medical journals (two American: NEJM and JAMA; and two British: The Lancet and The BMJ) in 22 newspapers. Results: British and American newspapers cited some of the four analysed medical journals about three times a week in 2008–2015 (weekly mean 3.2 and 2.7 citations, respectively); the newspapers from other Western countries did so about once a week (weekly mean 1.1), and those from the Rest of the World cited them about once a month (monthly mean 1.1). The New York Times cited above all other newspapers (weekly mean 4.7). The analysis showed the existence of three national citation patterns in the daily press: American newspapers cited mostly American journals (70.0% of citations), British newspapers cited mostly British journals (86.5%) and the rest of the analysed press cited more British journals than American ones. The Lancet was the most cited journal in the press of almost all Western countries outside the USA and the UK. Multivariate correspondence analysis confirmed the national patterns and showed that over 85% of the citation data variability is retained in just one single new variable: the national dimension. Conclusion: British and American newspapers are the ones that cite the four analysed medical journals more often, showing a domestic preference for their respective national journals; non-British and non-American newspapers show a common international citation pattern., Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Scoping review on interventions to improve adherence to reporting guidelines in health research

  • Blanco, David
  • Altman, Doug
  • Moher , David
  • Boutron, Isabelle
  • Kirkham, Jamie
  • Cobo, Erik
This scoping review belongs to the ESR 14 research project from the Methods in Research on Research (MiRoR) project (http://miror-ejd.eu/), which has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 676207. DM is supported through a University Research Chair (University of Ottawa)., Objectives: The goal of this study is to identify, analyse and classify interventions to improve adherence to reporting guidelines in order to obtain a wide picture of how the problem of enhancing the completeness of reporting of biomedical literature has been tackled so far. Design Scoping review. Search strategy We searched the MEDLINE, EMBASE and Cochrane Library databases and conducted a grey literature search for (1) studies evaluating interventions to improve adherence to reporting guidelines in health research and (2) other types of references describing interventions that have been performed or suggested but never evaluated. The characteristics and effect of the evaluated interventions were analysed. Moreover, we explored the rationale of the interventions identified and determined the existing gaps in research on the evaluation of interventions to improve adherence to reporting guidelines. Results: 109 references containing 31 interventions (11 evaluated) were included. These were grouped into five categories: (1) training on the use of reporting guidelines, (2) improving understanding, (3) encouraging adherence, (4) checking adherence and providing feedback, and (5) involvement of experts. Additionally, we identified lack of evaluated interventions (1) on training on the use of reporting guidelines and improving their understanding, (2) at early stages of research and (3) after the final acceptance of the manuscript. Conclusions: This scoping review identified a wide range of strategies to improve adherence to reporting guidelines that can be taken by different stakeholders. Additional research is needed to assess the effectiveness of many of these interventions., info:eu-repo/semantics/publishedVersion
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Effect of an editorial intervention to improve the completeness of reporting of randomised trials: a randomised controlled trial

  • Blanco de Tena Davila, David
  • Cobo Valeri, Erik
Objective To evaluate the impact of an editorial intervention to improve completeness of reporting of reports of randomised trials. Design Randomised controlled trial (RCT). Setting BMJ Open’s quality improvement programme. Participants 24 manuscripts describing RCTs. Interventions We used an R Shiny application to randomise manuscripts (1:1 allocation ratio, blocks of 4) to the intervention (n=12) or control (n=12) group. The intervention was performed by a researcher with expertise in the content of the Consolidated Standards of Reporting Trials (CONSORT) and consisted of an evaluation of completeness of reporting of eight core CONSORT items using the submitted checklist to locate information, and the production of a report containing specific requests for authors based on the reporting issues found, provided alongside the peer review reports. The control group underwent the usual peer review. Outcomes The primary outcome is the number of adequately reported items (0–8 scale) in the revised manuscript after the first round of peer review. The main analysis was intention-to-treat (n=24), and we imputed the scores of lost to follow-up manuscripts (rejected after peer review and not resubmitted). The secondary outcome is the proportion of manuscripts where each item was adequately reported. Two blinded reviewers assessed the outcomes independently and in duplicate and solved disagreements by consensus. We also recorded the amount of time to perform the intervention. Results Manuscripts in the intervention group (mean: 7.01; SD: 1.47) were more completely reported than those in the control group (mean: 5.68; SD: 1.43) (mean difference 1.43, 95% CI 0.31 to 2.58). We observed the main differences in items 6a (outcomes), 9 (allocation concealment mechanism), 11a (blinding) and 17a (outcomes and estimation). The mean time to perform the intervention was 87 (SD 42) min. Conclusions We demonstrated the benefit of involving a reporting guideline expert in the editorial process. Improving the completeness of RCTs is essential to enhance their usability. Trial registration number NCT03751878., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

The STROBE extensions: protocol for a qualitative assessment of content and a survey of endorsement

  • Sharp, Melissa K.
  • Utropia, Ana
  • Gómez Melis, Guadalupe
  • Cobo Valeri, Erik
  • Wager, Elisabeth
  • Hren, Darko
Introduction The STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) Statement was developed in response to inadequate reporting of observational studies. In recent years, several extensions to STROBE have been created to provide more nuanced field-specific guidance for authors. The content and the prevalence of extension endorsement have not yet been assessed. Accordingly, there are two aims: (1) to classify changes made in the extensions to identify strengths and weaknesses of the original STROBE checklist and (2) to determine the prevalence and typology of endorsement by journals in fields related to extensions. Methods and analysis Two independent researchers will assess additions in each extension. Additions will be coded as â field specific' (FS) or â not field specific' (NFS). FS is defined as particularly relevant information for a single field and guidance provided generally cannot be extrapolated beyond that field. NFS is defined as information that reflects epidemiological or methodological tenets and can be generalised to most, if not all, types of observational research studies. Intraclass correlation will be calculated to measure reviewers' concordance. On disagreement, consensus will be sought. Individual additions will be grouped by STROBE checklist items to identify the frequency and distribution of changes. Journals in fields related to extensions will be identified through National Library of Medicine PubMed Broad Subject Terms, screened for eligibility and further distilled via Ovid MEDLINE® search strategies for observational studies. Text describing endorsement will be extracted from each journal's website. A classification scheme will be created for endorsement types and the prevalence of endorsement will be estimated. Analyses will use NVivo V.11 and SAS University Edition. Ethics and dissemination This study does not require ethical approval as it does not involve human participants. This study has been preregistered on Open Science Framework., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Are CONSORT checklists submitted by authors adequately reflecting what information is actually reported in published papers?

  • Blanco de Tena Davila, David
  • Biganne, Alice M.
  • Cobo Valeri, Erik
Background: Compulsory submission of a checklist from the relevant reporting guideline is one of the most widespread journal requirements aiming to improve completeness of reporting. However, the current suboptimal levels of adherence to reporting guidelines observed in the literature may indicate that this journal policy is not having a significant effect. Findings: We explored whether authors provided the appropriate CONSORT checklist extension for their study and whether there were inconsistencies between what authors claimed on the submitted checklist and what was actually reported in the published paper. We randomly selected 12 randomized trials from three journals that provide the originally submitted checklist and analyzed six core CONSORT items. Only one paper used the appropriate checklist extension and had no inconsistencies between what was claimed in the submitted checklist and what was reported in the published paper. Conclusion: Journals should take further actions to take full advantage of the requirement for the submission of fulfilled CONSORT checklists, thus ensuring that these checklists reflect what is reported in the manuscript., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

National citation patterns of NEJM, The Lancet, JAMA and The BMJ in the lay press: a quantitative content analysis

  • Casino, Gonzalo
  • Rius Carrasco, Roser
  • Cobo Valeri, Erik
Objectives: To analyse the total number of newspaper articles citing the four leading general medical journals and to describe national citation patterns. Design: Quantitative content analysis. Setting/sample: Full text of 22 general newspapers in 14 countries over the period 2008–2015, collected from LexisNexis. The 14 countries have been categorised into four regions: the USA, the UK, Western World (European countries other than the UK, and Australia, New Zealand and Canada) and Rest of the World (other countries). Main outcome measure Press citations of four medical journals (two American: NEJM and JAMA; and two British: The Lancet and The BMJ) in 22 newspapers. Results: British and American newspapers cited some of the four analysed medical journals about three times a week in 2008–2015 (weekly mean 3.2 and 2.7 citations, respectively); the newspapers from other Western countries did so about once a week (weekly mean 1.1), and those from the Rest of the World cited them about once a month (monthly mean 1.1). The New York Times cited above all other newspapers (weekly mean 4.7). The analysis showed the existence of three national citation patterns in the daily press: American newspapers cited mostly American journals (70.0% of citations), British newspapers cited mostly British journals (86.5%) and the rest of the analysed press cited more British journals than American ones. The Lancet was the most cited journal in the press of almost all Western countries outside the USA and the UK. Multivariate correspondence analysis confirmed the national patterns and showed that over 85% of the citation data variability is retained in just one single new variable: the national dimension. Conclusion: British and American newspapers are the ones that cite the four analysed medical journals more often, showing a domestic preference for their respective national journals; non-British and non-American newspapers show a common international citation pattern., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Development of ARCADIA: a tool for assessing the quality of peer-review reports in biomedical research

  • Superchi, Cecilia
  • Hren, Darko
  • Blanco de Tena Davila, David
  • Rius Carrasco, Roser
  • Recchioni, Alessandro
  • Boutron, Isabelle
  • González Alastrué, José Antonio
Objective To develop a tool to assess the quality of peer- review reports in biomedical research. Methods We conducted an online survey intended for biomedical editors and authors. The survey aimed to (1) determine if participants endorse the proposed definition of peer- review report quality; (2) identify the most important items to include in the final version of the tool and (3) identify any missing items. Participants rated on a 5- point scale whether an item should be included in the tool and they were also invited to comment on the importance and wording of each item. Principal component analysis was performed to examine items redundancy and a general inductive approach was used for qualitative data analysis. Results A total of 446 biomedical editors and authors participated in the survey. Participants were mainly male (65.9%), middle- aged (mean=50.3, SD=13) and with PhD degrees (56.4%). The majority of participants (84%) agreed on the definition of peer- review report quality we proposed. The 20 initial items included in the survey questionnaire were generally highly rated with a mean score ranging from 3.38 (SD=1.13) to 4.60 (SD=0.69) (scale 1–5). Participants suggested 13 items that were not included in the initial list of items. A steering committee composed of five members with different expertise discussed the selection of items to include in the final version of the tool. The final checklist includes 14 items encompassed in five domains (Importance of the study, Robustness of the study methods, Interpretation and discussion of the study results, Reporting and transparency of the manuscript, Characteristics of peer reviewer’s comments). Conclusion Assessment of Review reports with a Checklist Available to eDItors and Authors tool could be used regularly by editors to evaluate the reviewers’ work, and also as an outcome when evaluating interventions to improve the peer- review process., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Effect of an editorial intervention to improve the completeness of reporting of randomised trials: a randomised controlled trial

  • Blanco de Tena Davila, David
  • Schroter, Sara
  • Aldcroft, Adrian
  • Moher, David
  • Boutron, Isabelle
  • Kirkham, Jamie J.
  • Cobo, Erik
This study is part of the ESR 14 research project from the Methods in Research on Research (MiRoR) project (http://miror-ejd.eu/), which has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement no 676 207. DM is supported through a University Research Chair (University of Ottawa). The sponsor and the funding source had no role in the design of this study and will not have any role during its execution, analyses, interpretation of the data or decision to submit results., Objective To evaluate the impact of an editorial intervention to improve completeness of reporting of reports of randomised trials. Design Randomised controlled trial (RCT). Setting BMJ Open’s quality improvement programme. Participants 24 manuscripts describing RCTs. Interventions We used an R Shiny application to randomise manuscripts (1:1 allocation ratio, blocks of 4) to the intervention (n=12) or control (n=12) group. The intervention was performed by a researcher with expertise in the content of the Consolidated Standards of Reporting Trials (CONSORT) and consisted of an evaluation of completeness of reporting of eight core CONSORT items using the submitted checklist to locate information, and the production of a report containing specific requests for authors based on the reporting issues found, provided alongside the peer review reports. The control group underwent the usual peer review. Outcomes The primary outcome is the number of adequately reported items (0–8 scale) in the revised manuscript after the first round of peer review. The main analysis was intention-to-treat (n=24), and we imputed the scores of lost to follow-up manuscripts (rejected after peer review and not resubmitted). The secondary outcome is the proportion of manuscripts where each item was adequately reported. Two blinded reviewers assessed the outcomes independently and in duplicate and solved disagreements by consensus. We also recorded the amount of time to perform the intervention. Results Manuscripts in the intervention group (mean: 7.01; SD: 1.47) were more completely reported than those in the control group (mean: 5.68; SD: 1.43) (mean difference 1.43, 95% CI 0.31 to 2.58). We observed the main differences in items 6a (outcomes), 9 (allocation concealment mechanism), 11a (blinding) and 17a (outcomes and estimation). The mean time to perform the intervention was 87 (SD 42) min. Conclusions We demonstrated the benefit of involving a reporting guideline expert in the editorial process. Improving the completeness of RCTs is essential to enhance their usability., info:eu-repo/semantics/publishedVersion
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

A survey exploring biomedical editors’ perceptions of editorial interventions to improve adherence to reporting guidelines

  • Blanco, David
  • Hren, Darko
  • Kirkham, Jamie
  • Cobo, Erik
  • Schroter, Sara
Background: Improving the completeness of reporting of biomedical research is essential for improving its usability. For this reason, hundreds of reporting guidelines have been created in the last few decades but adherence to these remains suboptimal. This survey aims to inform future evaluations of interventions to improve adherence to reporting guidelines. In particular, it gathers editors’ perceptions of a range of interventions at various stages in the editorial process. Methods: We surveyed biomedical journal editors that were knowledgeable about this topic. The questionnaire included open and closed questions that explored (i) the current practice of their journals, (ii) their perceptions of the ease of implementation of different interventions and the potential effectiveness of these at improving adherence to reporting guidelines, (iii) the barriers and facilitators associated with these interventions, and (iv) suggestions for future interventions and incentives. Results: Of the 99 editors invited, 24 (24%) completed the survey. Involving trained editors or administrative staff was deemed the potentially most effective intervention but, at the same time, it was considered moderately difficult to implement due to logistic and resource issues. Participants believed that checking adherence to guidelines goes beyond the role of peer reviewers and were concerned that the quality of peer review could be compromised. Reviewers are generally not expected to focus on reporting issues but on providing an expert view on the importance, novelty, and relevance of the manuscript. Journals incentivising adherence, and publishers and medical institutions encouraging journals to take action to boost adherence were two recurrent themes. Conclusions: Biomedical journal editors generally believed that engaging trained professionals would be the most effective, yet resource intensive, editorial intervention. Also, they thought that peer reviewers should not be asked to check RGs. Future evaluations of interventions can take into account the barriers, facilitators, and incentives described in this survey, info:eu-repo/semantics/publishedVersion
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Scoping review on interventions to improve adherence to reporting guidelines in health research

  • Blanco de Tena Davila, David
  • Altman, D.G.
  • Moher, David
  • Boutron, Isabelle
  • Kirkham, Jamie
  • Cobo Valeri, Erik
Objectives The goal of this study is to identify, analyse and classify interventions to improve adherence to reporting guidelines in order to obtain a wide picture of how the problem of enhancing the completeness of reporting of biomedical literature has been tackled so far. Design Scoping review. Search strategy We searched the MEDLINE, EMBASE and Cochrane Library databases and conducted a grey literature search for (1) studies evaluating interventions to improve adherence to reporting guidelines in health research and (2) other types of references describing interventions that have been performed or suggested but never evaluated. The characteristics and effect of the evaluated interventions were analysed. Moreover, we explored the rationale of the interventions identified and determined the existing gaps in research on the evaluation of interventions to improve adherence to reporting guidelines. Results 109 references containing 31 interventions (11 evaluated) were included. These were grouped into five categories: (1) training on the use of reporting guidelines, (2) improving understanding, (3) encouraging adherence, (4) checking adherence and providing feedback, and (5) involvement of experts. Additionally, we identified lack of evaluated interventions (1) on training on the use of reporting guidelines and improving their understanding, (2) at early stages of research and (3) after the final acceptance of the manuscript. Conclusions This scoping review identified a wide range of strategies to improve adherence to reporting guidelines that can be taken by different stakeholders. Additional research is needed to assess the effectiveness of many of these interventions, Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Taking advantage of unexpected WebCONSORT results

  • Cobo Valeri, Erik
  • González Alastrué, José Antonio
To estimate treatment effects, trials are initiated by randomising patients to the interventions under study and finish by comparing patient evolution. In order to improve the trial report, the CONSORT statement provides authors and peer reviewers with a guide of the essential items that would allow research replication. Additionally, WebCONSORT aims to facilitate author reporting by providing the items from the different CONSORT extensions that are relevant to the trial being reported. WebCONSORT has been estimated to improve the proportion of reported items by 0.04 (95% CI, –0.02 to 0.10), interpreted as “no important difference”, in accordance with the scheduled desired scenario of a 0.15 effect size improvement. However, in a non-scheduled analysis, it was found that, despite clear instructions, around a third of manuscripts selected for trials by the editorial staff were not actually randomised trials. We argue that surprises benefit science, and that further research should be conducted in order to improve the performance of editorial staff., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Adherence to reporting guidelines increases the number of citations: the argument for including a methodologist in the editorial process and peer-review

  • Vilaró, Marta
  • Cortés, Jordi
  • Selva O'Callaghan, Albert
  • Urrutia, Agustín
  • Ribera, Josep Maria
  • Cardellach, Francesc
  • Basagaña, Xavier
  • Elmore, Matthew
  • Vilardell, Miquel
  • Altman, Douglas G.
  • González, José Antonio
  • Cobo, Erik
Background: From 2005 to 2010, we conducted 2 randomized studies on a journal (Medicina Clínica), where we took manuscripts received for publication and randomly assigned them to either the standard editorial process or to additional processes. Both studies were based on the use of methodological reviewers and reporting guidelines (RG). Those interventions slightly improved the items reported on the Manuscript Quality Assessment Instrument (MQAI), which assesses the quality of the research report. However, masked evaluators were able to guess the allocated group in 62% (56/90) of the papers, thus presenting a risk of detection bias. In this post-hoc study, we analyse whether those interventions that were originally designed for improving the completeness of manuscript reporting may have had an effect on the number of citations, which is the measured outcome that we used. Methods: Masked to the intervention group, one of us used the Web of Science (WoS) to quantify the number of citations that the participating manuscripts received up December 2016. We calculated the mean citation ratio between intervention arms and then quantified the uncertainty of it by means of the Jackknife method, which avoids assumptions about the distribution shape. Results: Our study included 191 articles (99 and 92, respectively) from the two previous studies, which all together received 1336 citations. In both studies, the groups subjected to additional processes showed higher averages, standard deviations and annual rates. The intervention effect was similar in both studies, with a combined estimate of a 43% (95% CI: 3 to 98%) increase in the number of citations. Conclusions: We interpret that those effects are driven mainly by introducing into the editorial process a senior methodologist to find missing RG items. Those results are promising, but not definitive due to the exploratory nature of the study and some important caveats such as: the limitations of using the number of citations as a measure of scientific impact; and the fact that our study is based on a single journal. We invite journals to perform their own studies to ascertain whether or not scientific repercussion is increased by adhering to reporting guidelines and further involving statisticians in the editorial process.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Adherence to reporting guidelines increases the number of citations: the argument for including a methodologist in the editorial process and peer-review

  • Vilaró, Marta
  • Cortés Martínez, Jordi
  • Selva O’Callaghan, Albert
  • Urrutia, Agustí
  • Ribera, Josep Maria
  • Cardellach López, Francesc
  • Basagaña Flores, Xavier
  • Elmore, Matthew
  • Vilardell, Miquel
  • Altman, D.G.
  • González Alastrué, José Antonio
  • Cobo Valeri, Erik
From 2005 to 2010, we conducted 2 randomized studies on a journal (Medicina Clínica), where we took manuscripts received for publication and randomly assigned them to either the standard editorial process or to additional processes. Both studies were based on the use of methodological reviewers and reporting guidelines (RG). Those interventions slightly improved the items reported on the Manuscript Quality Assessment Instrument (MQAI), which assesses the quality of the research report. However, masked evaluators were able to guess the allocated group in 62% (56/90) of the papers, thus presenting a risk of detection bias. In this post-hoc study, we analyse whether those interventions that were originally designed for improving the completeness of manuscript reporting may have had an effect on the number of citations, which is the measured outcome that we used., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Adherence to reporting guidelines increases the number of citations: the argument for including a methodologist in the editorial process and peer-review

  • Vilaró, Marta
  • Cortés, Jordi
  • Selva-O'Callaghan, Albert
  • Urrutia, Agustín
  • Ribera, Josep Maria
  • Cardellach, Francesc
  • Basagaña Flores, Xavier
  • Elmore, Matthew
  • Vilardell, Miquel
  • Altman, Douglas
  • González, José-Antonio
  • Cobo Valeri, Erik
Background: From 2005 to 2010, we conducted 2 randomized studies on a journal (Medicina Clínica), where we took manuscripts received for publication and randomly assigned them to either the standard editorial process or to additional processes. Both studies were based on the use of methodological reviewers and reporting guidelines (RG). Those interventions slightly improved the items reported on the Manuscript Quality Assessment Instrument (MQAI), which assesses the quality of the research report. However, masked evaluators were able to guess the allocated group in 62% (56/90) of the papers, thus presenting a risk of detection bias. In this post-hoc study, we analyse whether those interventions that were originally designed for improving the completeness of manuscript reporting may have had an effect on the number of citations, which is the measured outcome that we used. Methods: Masked to the intervention group, one of us used the Web of Science (WoS) to quantify the number of citations that the participating manuscripts received up December 2016. We calculated the mean citation ratio between intervention arms and then quantified the uncertainty of it by means of the Jackknife method, which avoids assumptions about the distribution shape. Results: Our study included 191 articles (99 and 92, respectively) from the two previous studies, which all together received 1336 citations. In both studies, the groups subjected to additional processes showed higher averages, standard deviations and annual rates. The intervention effect was similar in both studies, with a combined estimate of a 43% (95% CI: 3 to 98%) increase in the number of citations. Conclusions: We interpret that those effects are driven mainly by introducing into the editorial process a senior methodologist to find missing RG items. Those results are promising, but not definitive due to the exploratory nature of the study and some important caveats such as: the limitations of using the number of citations as a measure of scientific impact; and the fact that our study is based on a single journal. We invite journals to perform their own studies to ascertain whether or not scientific repercussion is increased by adhering to reporting guidelines and further involving statisticians in the editorial process., DGA, JAG and EC belong to MiRoR (Methods in Research on Research), which has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 676207. EC and JC are partially supported by the grants MTM2015–64465-C2–1-R (MINECO/FEDER, UE) from the Secretaría de Estado de Investigación, Desarrollo e Innovación del Ministerio de Economía y Competitividad (Spain) and through grant 2017 SGR 622 from the Departament d’Economia i Coneixement de la Generalitat de Catalunya (Spain). The funding bodies had no role in the study design, data collection and analysis, results interpretation, or in writing the manuscript.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

A scoping review protocol on the roles and tasks of peer reviewers in the manuscript review process in biomedical journals

  • Glonti, Ketevan
  • Cauchi, Daniel
  • Cobo Valeri, Erik
  • Boutron, Isabelle
  • Moher, David
  • Hren, Darko
Introduction The primary functions of peer reviewers are poorly defined. Thus far no body of literature has systematically identified the roles and tasks of peer reviewers of biomedical journals. A clear establishment of these can lead to improvements in the peer review process. The purpose of this scoping review is to determine what is known on the roles and tasks of peer reviewers. Methods We will use the methodological framework first proposed by Arksey and O'Malley and subsequently adapted by Levac et al and the Joanna Briggs Institute. The scoping review will include all study designs, as well as editorials, commentaries and grey literature. The following eight electronic databases will be searched (from inception to May 2017): Cochrane Library, Cumulative Index to Nursing and Allied Health Literature, Educational Resources Information Center, EMBASE, MEDLINE, PsycINFO, Scopus and Web of Science. Two reviewers will use inclusion and exclusion criteria based on the 'Population-Concept-Context' framework to independently screen titles and abstracts of articles considered for inclusion. Full-text screening of relevant eligible articles will also be carried out by two reviewers. The search strategy for grey literature will include searching in websites of existing networks, biomedical journal publishers and organisations that offer resources for peer reviewers. In addition we will review journal guidelines to peer reviewers on how to perform the manuscript review. Journals will be selected using the 2016 journal impact factor. We will identify and assess the top five, middle five and lowest-ranking five journals across all medical specialties. Ethics and dissemination This scoping review will undertake a secondary analysis of data already collected and does not require ethical approval. The results will be disseminated through journals and conferences targeting stakeholders involved in peer review in biomedical research., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Statistical models to predict recruitment in clinical trials were rarely used by statisticians in UK and European networks

  • Gkioni, Efstathia
  • Dodd, Susanna
  • Rius Carrasco, Roser
  • Gamble, Carrol
© 2020. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/, Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Biomedical doctoral students’ research practices when facing dilemmas: two vignette-based randomized control trials

  • Nguyen, V.T.
  • Sharp, M. K.
  • Superchi, C.
  • Baron, G.
  • Glonti, K.
  • Blanco de Tena-Dávila, Elena
  • Olsen, M.
  • Tat, T.T Vo
  • Olarte Parra, C.
  • Névéol, A.
  • Hren, D.
  • Ravaud, P.
  • Boutron, I.
Tis project received funding from the European Union’s Horizon 2020 research and innovation program under the Marie Sklodowska-Curie grant agreement No 676207. Tis project also received funding from the Center d’Epidémiologie Clinique APHP, Université Paris Cité., Our aim was to describe the research practices of doctoral students facing a dilemma to research integrity and to assess the impact of inappropriate research environments, i.e. exposure to (a) a post-doctoral researcher who committed a Detrimental Research Practice (DRP) in a similar situation and (b) a supervisor who did not oppose the DRP. We conducted two 2-arm, parallel-group randomized controlled trials. We created 10 vignettes describing a realistic dilemma with two alternative courses of action (good practice versus DRP). 630 PhD students were randomized through an online system to a vignette (a) with (n = 151) or without (n = 164) exposure to a post-doctoral researcher; (b) with (n = 155) or without (n = 160) exposure to a supervisor. The primary outcome was a score from − 5 to + 5, where positive scores indicated the choice of DRP and negative scores indicated good practice. Overall, 37% of unexposed participants chose to commit DRP with important variation across vignettes (minimum 10%; maximum 66%). The mean difference [95%CI] was 0.17 [− 0.65 to 0.99;], p = 0.65 when exposed to the post-doctoral researcher, and 0.79 [− 0.38; 1.94], p = 0.16, when exposed to the supervisor. In conclusion, we did not find evidence of an impact of postdoctoral researchers and supervisors on student research practices., info:eu-repo/semantics/publishedVersion
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Are CONSORT checklists submitted by authors adequately reflecting what information is actually reported in published papers?

  • Blanco, David
  • Biggane, Alice M.
  • Cobo, Erik
  • MiRoR network
This study is part of the June 2017 Journal Club, organized by David Blanco, Alice Biggane and Erik Cobo within the MiRoR project. This project received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement, no 676207., Background: Compulsory submission of a checklist from the relevant reporting guideline is one of the most widespread journal requirements aiming to improve completeness of reporting. However, the current suboptimal levels of adherence to reporting guidelines observed in the literature may indicate that this journal policy is not having a significant effect. Findings: We explored whether authors provided the appropriate CONSORT checklist extension for their study and whether there were inconsistencies between what authors claimed on the submitted checklist and what was actually reported in the published paper. We randomly selected 12 randomized trials from three journals that provide the originally submitted checklist and analyzed six core CONSORT items. Only one paper used the appropriate checklist extension and had no inconsistencies between what was claimed in the submitted checklist and what was reported in the published paper. Conclusion: Journals should take further actions to take full advantage of the requirement for the submission of fulfilled CONSORT checklists, thus ensuring that these checklists reflect what is reported in the manuscript., info:eu-repo/semantics/publishedVersion
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Interventions to improve adherence to reporting guidelines in health research: a scoping review protocol

  • Blanco de Tena Davila, David
  • Kirkham, Jamie
  • Altman, D.G.
  • Moher, David
  • Boutron, Isabelle
  • Cobo Valeri, Erik
There is evidence that the use of some reporting guidelines, such as the Consolidated Standards for Reporting Trials, is associated with improved completeness of reporting in health research. However, the current levels of adherence to reporting guidelines are suboptimal. Over the last few years, several actions aiming to improve compliance with reporting guidelines have been taken and proposed. We will conduct a scoping review of interventions to improve adherence to reporting guidelines in health research that have been evaluated or suggested, in order to inform future interventions., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Development of ARCADIA: a tool for assessing the quality of peer-review reports in biomedical research

  • Superchi, Cecilia
  • Hren, Darko
  • Blanco de Tena Davila, David
  • Rius, Roser
  • Recchioni, Alessandro
  • Boutron, Isabelle
  • González, José Antonio
This project was supported by the European Union’s Horizon 2020 Research and Innovation Programme under the Marie Sklodowska-Curie grant agreement no 676207., Objective: To develop a tool to assess the quality of peer-review reports in biomedical research. Methods We conducted an online survey intended for biomedical editors and authors. The survey aimed to (1) determine if participants endorse the proposed definition of peer-review report quality; (2) identify the most important items to include in the final version of the tool and (3) identify any missing items. Participants rated on a 5-point scale whether an item should be included in the tool and they were also invited to comment on the importance and wording of each item. Principal component analysis was performed to examine items redundancy and a general inductive approach was used for qualitative data analysis. Results: A total of 446 biomedical editors and authors participated in the survey. Participants were mainly male (65.9%), middle-aged (mean=50.3, SD=13) and with PhD degrees (56.4%). The majority of participants (84%) agreed on the definition of peer-review report quality we proposed. The 20 initial items included in the survey questionnaire were generally highly rated with a mean score ranging from 3.38 (SD=1.13) to 4.60 (SD=0.69) (scale 1–5). Participants suggested 13 items that were not included in the initial list of items. A steering committee composed of five members with different expertise discussed the selection of items to include in the final version of the tool. The final checklist includes 14 items encompassed in five domains (Importance of the study, Robustness of the study methods, Interpretation and discussion of the study results, Reporting and transparency of the manuscript, Characteristics of peer reviewer’s comments). Conclusion: Assessment of Review reports with a Checklist Available to eDItors and Authors tool could be used regularly by editors to evaluate the reviewers’ work, and also as an outcome when evaluating interventions to improve the peer-review process., info:eu-repo/semantics/publishedVersion
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Tools used to assess the quality of peer review reports: a methodological systematic review

  • Superchi, Cecilia
  • González Alastrué, José Antonio
  • Solà, Ivan
  • Cobo Valeri, Erik
  • Hren, Darko
  • Boutron, Isabelle
Background A strong need exists for a validated tool that clearly defines peer review report quality in biomedical research, as it will allow evaluating interventions aimed at improving the peer review process in well-performed trials. We aim to identify and describe existing tools for assessing the quality of peer review reports in biomedical research. Methods We conducted a methodological systematic review by searching PubMed, EMBASE (via Ovid) and The Cochrane Methodology Register (via The Cochrane Library) as well as Google® for all reports in English describing a tool for assessing the quality of a peer review report in biomedical research. Data extraction was performed in duplicate using a standardized data extraction form. We extracted information on the structure, development and validation of each tool. We also identified quality components across tools using a systematic multi-step approach and we investigated quality domain similarities among tools by performing hierarchical, complete-linkage clustering analysis. Results We identified a total number of 24 tools: 23 scales and 1 checklist. Six tools consisted of a single item and 18 had several items ranging from 4 to 26. None of the tools reported a definition of ‘quality’. Only 1 tool described the scale development and 10 provided measures of validity and reliability. Five tools were used as an outcome in a randomized controlled trial (RCT). Moreover, we classified the quality components of the 18 tools with more than one item into 9 main quality domains and 11 subdomains. The tools contained from two to seven quality domains. Some domains and subdomains were considered in most tools such as the detailed/thorough (11/18) nature of reviewer’s comments. Others were rarely considered, such as whether or not the reviewer made comments on the statistical methods (1/18). Conclusion Several tools are available to assess the quality of peer review reports; however, the development and validation process is questionable and the concepts evaluated by these tools vary widely. The results from this study and from further investigations will inform the development of a new tool for assessing the quality of peer review reports in biomedical research., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Interventions to improve adherence to reporting guidelines in health research: a scoping review protocol

  • Blanco, David
  • Kirkham, Jamie J.
  • Altman, Douglas G.
  • Moher, David
  • Boutron, Isabelle
  • Cobo, Erik
This scoping review belongs to the ESR 14 research project from the Methods in Research on Research (MiRoR) project, which has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 676207., There is evidence that the use of some reporting guidelines, such as the Consolidated Standards for Reporting Trials, is associated with improved completeness of reporting in health research. However, the current levels of adherence to reporting guidelines are suboptimal. Over the last few years, several actions aiming to improve compliance with reporting guidelines have been taken and proposed. We will conduct a scoping review of interventions to improve adherence to reporting guidelines in health research that have been evaluated or suggested, in order to inform future interventions. Methods and analysis Our review will follow the Joanna Briggs Institute scoping review methods manual. We will search for relevant studies in MEDLINE, EMBASE and Cochrane Library databases. Moreover, we will carry out lateral searches from the reference lists of the included studies, as well as from the lists of articles citing the included ones. One reviewer will screen the full list, which will be randomly split into two halves and independently screened by the other two reviewers. Two reviewers will perform data extraction independently. Discrepancies will be solved through discussion. In addition, this search strategy will be supplemented by a grey literature search. The interventions found will be classified as assessed or suggested, as well as according to different criteria, in relation to their target (journal policies, journal editors, authors, reviewers, funders, ethical boards or others) or the research stage at which they are performed (design, conducting, reporting or peer review). Descriptive statistical analysis will be performed. Ethics and dissemination A paper summarising the findings from this review will be published in a peer-reviewed journal. This scoping review will contribute to a better understanding and a broader perspective on how the problem of adhering better to reporting guidelines has been tackled so far. This could be a major first step towards developing future strategies to improve compliance with reporting guidelines in health research., info:eu-repo/semantics/publishedVersion
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

A systematic review describes models for recruitment prediction at the design stage of a clinical trial

  • Gkioni, Efstathia
  • Rius Carrasco, Roser
  • Dodd, Susanna
  • Gamble, Carrol
© 2019 Elsevier. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/, Patient recruitment in clinical trials is challenging with failure to recruit to time and target sample size common. This may be caused by unanticipated problems or by overestimation of the recruitment rate. This study is a systematic review of statistical models to predict recruitment at the design stage of clinical trials, The authors would like to thank Ms Van Nguyen Thu, PhD candidate, on the MiRoR project, and Professor Isabelle Boutron, Paris Descartes University, for their help in reviewing the manuscript before submission. This project is part of the MiRoR (Methods in Research on Research) Joint Doctoral Training Programme. The authors would like to acknowledge the MiRoR network for their support and guidance. The European Union's Horizon 2020 Marie Sklodowska Curie Innovative Training Networks–European Joint Doctorate (ITN-EJD) under grant agreement No. 676207, fund MiRoR., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Are CONSORT checklists submitted by authors adequately reflecting what information is actually reported in published papers?

  • Blanco, David
  • Biggane, Alice M.
  • Cobo, Erik
  • MiRoR network
This study is part of the June 2017 Journal Club, organized by David Blanco, Alice Biggane and Erik Cobo within the MiRoR project. This project received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement, no 676207., Background: Compulsory submission of a checklist from the relevant reporting guideline is one of the most widespread journal requirements aiming to improve completeness of reporting. However, the current suboptimal levels of adherence to reporting guidelines observed in the literature may indicate that this journal policy is not having a significant effect. Findings: We explored whether authors provided the appropriate CONSORT checklist extension for their study and whether there were inconsistencies between what authors claimed on the submitted checklist and what was actually reported in the published paper. We randomly selected 12 randomized trials from three journals that provide the originally submitted checklist and analyzed six core CONSORT items. Only one paper used the appropriate checklist extension and had no inconsistencies between what was claimed in the submitted checklist and what was reported in the published paper. Conclusion: Journals should take further actions to take full advantage of the requirement for the submission of fulfilled CONSORT checklists, thus ensuring that these checklists reflect what is reported in the manuscript., info:eu-repo/semantics/publishedVersion
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Using the STROBE statement: survey findings emphasized the role of journals in enforcing reporting guidelines

  • Sharp, Melissa K.
  • Bertizzolo, Lorenzo
  • Rius Carrasco, Roser
  • Wager, Elisabeth
  • Gómez Melis, Guadalupe
  • Hren, Darko
© 2019 Elsevier. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/, Objectives The objective of the study was to identify factors affecting the use of the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement, specifically authors’ attitudes toward and experiences with it. Study Design and Setting An online survey was distributed to authors of observational studies recruited via social media, personal network snowballing, and mass mailings using targeted search strategies. Data on demographics, awareness, motivators, and usage were collected in conjunction with a modified Unified Theory of Acceptance and Use of Technology (UTAUT) scale on which confirmatory factor analysis (CFA) was performed. Results One thousand fifteen participants completed the survey. Of these, 185 (18.2%) indicated they had never heard of STROBE nor used it previously, 195 (19.2%) had heard of it but never used it, and 635 (62.6%) had used it. Journals promoting STROBE were both key motivators and awareness mechanisms; peers and educational workshops were also important influencing factors to a lesser degree. The internal consistency of the modified UTAUT scale was strong (Cronbach's alpha = 0.94). CFA supported a four-factor model with 23 questions. Conclusion The endorsement of STROBE by journals is key to authors’ awareness and use of the guideline. We tested and validated our scale which can guide future research on reporting guidelines., The authors would like to acknowledge the Methods in Research on Research (MiRoR) network for their support and guidance. The authors would also like to thank Dr. Michail Kovanis for assistance gathering author emails and Ms. Alice Biggane for her assistance with the recruitment strategy., Peer Reviewed, Postprint (published version)
Dipòsit Digital de Documents de la UAB
Publication Artículo científico (JournalArticle)

Adherence to reporting guidelines increases the number of citations, the argument for including a methodologist in the editorial process and peer-review

  • Vilaró, Marta|||0000-0001-5692-0891
  • Cortés, Jordi|||0000-0002-3764-0795
  • Selva O'Callaghan, Albert|||0000-0003-2823-9761
  • Urrutia de Diego, Agustín|||0000-0002-7508-5091
  • Ribera, Jose-Maria|||0000-0003-1042-6024
  • Cardellach, Francesc|||0000-0003-3286-3436
  • Basagaña, X|||0000-0002-8457-1489
  • Elmore, Matthew
  • Vilardell, Miquel
  • Altman, Douglas G|||0000-0002-7183-4083
  • González, José-Antonio
  • Cobo, Erik
From 2005 to 2010, we conducted 2 randomized studies on a journal (Medicina Clínica), where we took manuscripts received for publication and randomly assigned them to either the standard editorial process or to additional processes. Both studies were based on the use of methodological reviewers and reporting guidelines (RG). Those interventions slightly improved the items reported on the Manuscript Quality Assessment Instrument (MQAI), which assesses the quality of the research report. However, masked evaluators were able to guess the allocated group in 62% (56/90) of the papers, thus presenting a risk of detection bias. In this post-hoc study, we analyse whether those interventions that were originally designed for improving the completeness of manuscript reporting may have had an effect on the number of citations, which is the measured outcome that we used. Masked to the intervention group, one of us used the Web of Science (WoS) to quantify the number of citations that the participating manuscripts received up December 2016. We calculated the mean citation ratio between intervention arms and then quantified the uncertainty of it by means of the Jackknife method, which avoids assumptions about the distribution shape. Our study included 191 articles (99 and 92, respectively) from the two previous studies, which all together received 1336 citations. In both studies, the groups subjected to additional processes showed higher averages, standard deviations and annual rates. The intervention effect was similar in both studies, with a combined estimate of a 43% (95% CI: 3 to 98%) increase in the number of citations. We interpret that those effects are driven mainly by introducing into the editorial process a senior methodologist to find missing RG items. Those results are promising, but not definitive due to the exploratory nature of the study and some important caveats such as: the limitations of using the number of citations as a measure of scientific impact; and the fact that our study is based on a single journal. We invite journals to perform their own studies to ascertain whether or not scientific repercussion is increased by adhering to reporting guidelines and further involving statisticians in the editorial process. The online version of this article (10.1186/s12874-019-0746-4) contains supplementary material, which is available to authorized users.