Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 9
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Lightweight transformers for clinical natural language processing

  • Rohanian, Omid
  • Nouriborji, Mohammadmahdi
  • Jauncey, Hannah
  • Kouchaki, Samaneh
  • Nooralahzadeh, Farhad
  • ISARIC Clinical Characterisation Group
  • Clifton, Lei
  • Merson, Laura
  • Clifton, David A.
Specialised pre-trained language models are becoming more frequent in Natural language Processing (NLP) since they can potentially outperform models trained on generic texts. BioBERT (Sanh et al., Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv: 1910.01108, 2019) and BioClinicalBERT (Alsentzer et al., Publicly available clinical bert embeddings. In Proceedings of the 2nd Clinical Natural Language Processing Workshop, pp. 72-78, 2019) are two examples of such models that have shown promise in medical NLP tasks. Many of these models are overparametrised and resource-intensive, but thanks to techniques like knowledge distillation, it is possible to create smaller versions that perform almost as well as their larger counterparts. In this work, we specifically focus on development of compact language models for processing clinical texts (i.e. progress notes, discharge summaries, etc). We developed a number of efficient lightweight clinical transformers using knowledge distillation and continual learning, with the number of parameters ranging from million to million. These models performed comparably to larger models such as BioBERT and ClinicalBioBERT and significantly outperformed other compact models trained on general or biomedical data. Our extensive evaluation was done across several standard datasets and covered a wide range of clinical text-mining tasks, including natural language inference, relation extraction, named entity recognition and sequence classification. To our knowledge, this is the first comprehensive study specifically focused on creating efficient and compact transformers for clinical NLP tasks. The models and code used in this study can be found on our Huggingface profile at https://huggingface.co/nlpie and Github page at https://github.com/nlpie-research/Lightweight-Clinical-Transformers, respectively, promoting reproducibility of our results., This work was made possible with the support of UK Foreign, Commonwealth and Development Office and Wellcome [225288/Z/22/Z]. Collection of data for the ISARIC Clinical Notes was made possible with the support of UK Foreign, Commonwealth and Development Office and Wellcome [215091/Z/18/Z, 222410/Z/21/Z, 225288/Z/22/Z, 220757/Z/20/Z and 222048/Z/20/Z] and the Bill & Melinda Gates Foundation [OPP1209135]; CIHR Coronavirus Rapid Research Funding Opportunity OV2170359 and was coordinated out of Sunnybrook Research Institute; was supported by endorsement of the Irish Critical Care- Clinical Trials Group, co-ordinated in Ireland by the Irish Critical Care- Clinical Trials Network at University College Dublin and funded by the Health Research Board of Ireland [CTN-2014-12]; grants from Rapid European COVID-19 Emergency Response research (RECOVER) [H2020 project 101003589] and European Clinical Research Alliance on Infectious Diseases (ECRAID) [965313]; Cambridge NIHR Biomedical Research Centre; Wellcome Trust fellowship [205228/Z/16/Z] and the National Institute for Health Research Health Protection Research Unit (HPRU) in Emerging and Zoonotic Infections (NIHR200907) at the University of Liverpool in partnership with Public Health England (PHE), in collaboration with Liverpool School of Tropical Medicine and the University of Oxford; the dedication and hard work of the Norwegian SARS-CoV-2 study team. Research Council of Norway grant no 312780 and a philanthropic donation from Vivaldi Invest A/S owned by Jon Stephenson von Tetzchner; PJMO was supported by the UK’s National Institute for Health Research (NIHR) via Imperial’s Biomedical Research Centre (NIHR Imperial BRC), Imperial’s Health Protection Research Unit in Respiratory Infections (NIHR HPRU RI), the Comprehensive Local Research Networks (CLRNs) and is an NIHR Senior Investigator (NIHR201385); Innovative Medicines Initiative Joint Undertaking under Grant Agreement No. 115523 COMBACTE, resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies, in-kind contribution; Stiftungsfonds zur Förderung der Bekämpfung der Tuberkulose und anderer Lungenkrankheiten of the City of Vienna; Project Number: APCOV22BGM; Australian Department of Health grant (3273191); Gender Equity Strategic Fund at University of Queensland, Artificial Intelligence for Pandemics (A14PAN) at University of Queensland, The Australian Research Council Centre of Excellence for Engineered Quantum Systems (EQUS, CE170100009), The Prince Charles Hospital Foundation, Australia; grants from Instituto de Salud Carlos III, Ministerio de Ciencia, Spain; Brazil, National Council for Scientific and Technological Development Scholarship number 303953/2018-7; the Firland Foundation, Shoreline, Washington, USA; The French COVID cohort (NCT04262921) is sponsored by INSERM and is funding by the REACTing (REsearch & ACtion emergING infectious diseases) consortium and by a grant of the French Ministry of Health (PHRC n 20-0424); the South Eastern Norway Health Authority and the Research Council of Norway; and a grant from the Oxford University COVID-19 Research Response fund (grant 0009109); Institute for Clinical Research (ICR), National Institutes of Health (NIH) supported by the Ministry of Health Malaysia; a grant from foundation Bevordering Onderzoek Franciscus.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

A multi-country analysis of COVID-19 hospitalizations by vaccination status

  • Gonçalves, Bronner P.
  • Martin-Loeches, Ignacio
  • Merson, Laura
  • ISARIC Clinical Characterisation Group
Background. Individuals vaccinated against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), when infected, can still develop disease that requires hospitalization. It remains unclear whether these patients differ from hospitalized unvaccinated patients with regard to presentation, coexisting comorbidities, and outcomes. Methods. Here, we use data from an international consortium to study this question and assess whether differences between these groups are context specific. Data from 83,163 hospitalized COVID-19 patients (34,843 vaccinated, 48,320 unvaccinated) from 38 countries were analyzed. Findings. While typical symptoms were more often reported in unvaccinated patients, comorbidities, including some associated with worse prognosis in previous studies, were more common in vaccinated patients. Considerable between-country variation in both in-hospital fatality risk and vaccinated-versus-unvaccinated difference in this outcome was observed. Conclusions. These findings will inform allocation of healthcare resources in future surges as well as design of longer-term international studies to characterize changes in clinical profile of hospitalized COVID-19 patients related to vaccination history., This work was also funded by grants from the National Institute for Health Research (NIHR; award CO-CIN-01), the Medical Research Council (MRC; grant MC_PC_19059), and the NIHR Health Protection Research Unit (HPRU) in Emerging and Zoonotic Infections at the University of Liverpool in partnership with Public Health England (PHE) (award 200907), NIHR HPRU in Respiratory Infections at Imperial College London with PHE (award 200927), Liverpool Experimental Cancer Medicine Centre (grant C18616/A25153), NIHR Biomedical Research Centre at Imperial College London (award ISBRC-1215-20013), and NIHR Clinical Research Network providing infrastructure support; CIHR Coronavirus Rapid Research Funding Opportunity OV2170359 and the coordination in Canada by Sunnybrook Research Institute; funding by the Health Research Board of Ireland (CTN-2014-12); the Rapid European COVID-19 Emergency Response research (RECOVER) (H2020 project 101003589) and European Clinical Research Alliance on Infectious Diseases (ECRAID) (965313); Cambridge NIHR Biomedical Research Centre (award NIHR203312); a Research Council of Norway grant no. 312780 and a philanthropic donation from Vivaldi Invest A/S owned by Jon Stephenson von Tetzchner; the Comprehensive Local Research Networks (CLRNs), of which P.J.M.O. is an NIHR Senior Investigator (NIHR201385); Stiftungsfonds zur Förderung der Bekämpfung der Tuberkulose und anderer Lungenkrankheiten of the City of Vienna, project number APCOV22BGM; funding from Medical University of Vienna, Department of Anaesthesia, Intensive Care Medicine and Pain Medicine; Italian Ministry of Health “Fondi Ricerca corrente–L1P6” to IRCCS Ospedale Sacro Cuore–Don Calabria; Australian Department of Health grant (3273191); Gender Equity Strategic Fund at University of Queensland, Artificial Intelligence for Pandemics (A14PAN) at University of Queensland, the Australian Research Council Centre of Excellence for Engineered Quantum Systems (EQUS, CE170100009), the Prince Charles Hospital Foundation, Australia; a grant from foundation Bevordering Onderzoek Franciscus; the South Eastern Norway Health Authority and the Research Council of Norway; Institute of Cancer Research (ICR), National Institutes of Health (NIH) supported by the Ministry of Health Malaysia; and the US DoD Armed Forces Health Surveillance Division, Global Emerging Infectious Diseases Branch to the US Naval Medical ResearchUnit no. TWO (NAMRU-2) (Work Unit #: P0153_21_N2). These authors would like to thank Vysnova Partners, Inc., for the management of this research project. The Lao-Oxford-Mahosot Hospital-Wellcome Trust Research Unit is funded by the Wellcome Trust; M.U.G.K. was supported by the Branco Weiss Fellowship, Google.org, the Oxford Martin School, the Rockefeller Foundation, and the European Union Horizon 2020 project MOOD (#874850). The contents of this publication are the sole responsibility of the authors and do not necessarily reflect the views of the European Commission. The investigators acknowledge the support of the COVID clinical management team, AIIMS, Rishikesh, India; the COVID-19 Clinical Management team, Manipal Hospital Whitefield, Bengaluru, India; the Liverpool School of Tropical Medicine and the University of Oxford; Imperial NIHR Biomedical Research Centre; the dedication and hard work of the Norwegian SARS-CoV-2 study team; endorsement of the Irish Critical Care Clinical Trials Group, co-ordination in Ireland by the Irish Critical Care Clinical Trials Network at University College Dublin; and preparedness work conducted by the Short Period Incidence Study of Severe Acute Respiratory Infection. This work uses data provided by patients and collected by the NHS as part of their care and support #DataSavesLives. The data used for this research were obtained from ISARIC4C. We are extremely grateful to the 2,648 frontline NHS clinical and research staff and volunteer medical students who collected these data in challenging circumstances and for the generosity of the patients and their families for their individual contributions in these difficult times. The COVID-19 Clinical Information Network (CO-CIN) data were collated by ISARIC4C Investigators. We also acknowledge the support of Jeremy J. Farrar and Nahoko Shindo.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

ISARIC-COVID-19 dataset: A prospective, standardized, global dataset of patients hospitalized with COVID-19

  • ISARIC Clinical Characterisation Group
  • Reyes, Luis F.
The International Severe Acute Respiratory and Emerging Infection Consortium (ISARIC) COVID-19 dataset is one of the largest international databases of prospectively collected clinical data on people hospitalized with COVID-19. This dataset was compiled during the COVID-19 pandemic by a network of hospitals that collect data using the ISARIC-World Health Organization Clinical Characterization Protocol and data tools. The database includes data from more than 705,000 patients, collected in more than 60 countries and 1,500 centres worldwide. Patient data are available from acute hospital admissions with COVID-19 and outpatient follow-ups. The data include signs and symptoms, pre-existing comorbidities, vital signs, chronic and acute treatments, complications, dates of hospitalization and discharge, mortality, viral strains, vaccination status, and other data. Here, we present the dataset characteristics, explain its architecture and how to gain access, and provide tools to facilitate its use., This work is part of the Grand Challenges ICODA pilot initiative, delivered by Health Data Research UK and funded by the Bill & Melinda Gates Foundation and the Minderoo Foundation. The philanthropic support of the donors to the University of Oxford’s COVID-19 Research Response Fund; UK Foreign, Commonwealth and Development Office and Wellcome [215091/Z/18/Z and 220757/Z/20/Z]; the Bill & Melinda Gates Foundation [OPP1209135]; the National Institute for Health Research (NIHR; award CO-CIN-01); the Medical Research Council (MRC; grant MC_PC_19059); the NIHR Health Protection Research Unit (HPRU) in Emerging and Zoonotic Infections at University of Liverpool in partnership with Public Health England (PHE)(award 200907); NIHR HPRU in Respiratory Infections at Imperial College London with PHE (award 200927); Liverpool Experimental Cancer Medicine Centre (grant C18616/A25153); NIHR Biomedical Research Centre at Imperial College London (award IS-BRC-1215-20013); NIHR Clinical Research Network (infrastructure support); CIHR Coronavirus Rapid Research Funding Opportunity OV2170359 and was coordinated out of Sunnybrook Research Institute; the endorsement of the Irish Critical Care- Clinical Trials Group, co-ordinated in Ireland by the Irish Critical Care- Clinical Trials Network at University College Dublin and funded by the Health Research Board of Ireland [CTN-2014-12]; Rapid European COVID-19 Emergency Response research (RECOVER) [H2020 project 101003589]; European Clinical Research Alliance on Infectious Diseases (ECRAID) [965313]; COVID clinical management team, AIIMS, Rishikesh, India; Cambridge NIHR Biomedical Research Centre; the dedication and hard work of the Groote Schuur Hospital Covid ICU Team; the Groote Schuur nursing and University of Cape Town registrar bodies coordinated by the Division of Critical Care at the University of Cape Town; Wellcome Trust fellowship [205228/Z/16/Z]; the Liverpool School of Tropical Medicine; the University of Oxford; the dedication and hard work of the Norwegian SARS-CoV-2 study team; the Research Council of Norway grant no 312780; a philanthropic donation from Vivaldi Invest A/S owned by Jon Stephenson von Tetzchner; Innovative Medicines Initiative Joint Undertaking under Grant Agreement No. 115523 COMBACTE, resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007–2013) and EFPIA companies, in-kind contribution; preparedness work conducted by the Short Period Incidence Study of Severe Acute Respiratory Infection; Stiftungsfonds zur Förderung der Bekämpfung der Tuberkulose und anderer Lungenkrankheiten of the City of Vienna, Project Number: APCOV22BGM; Italian Ministry of Health “Fondi Ricerca corrente–L1P6” to IRCCS Ospedale Sacro Cuore–Don Calabria; Australian Department of Health grant (3273191); Gender Equity Strategic Fund at University of Queensland; Artificial Intelligence for Pandemics (A14PAN) at University of Queensland; the Australian Research Council Centre of Excellence for Engineered Quantum Systems (EQUS, CE170100009); the Prince Charles Hospital Foundation, Australia; UK Medical Research Council Clinical Research Training Fellowship MR/V001671/1; Instituto de Salud Carlos III, Ministerio de Ciencia, Spain; Brazil, National Council for Scientific and Technological Development Scholarship number 303953/2018-7; Firland Foundation, Shoreline, Washington, USA; the French COVID cohort (NCT04262921) is sponsored by INSERM and is funding by the REACTing (REsearch & ACtion emergING infectious diseases) consortium and by a grant of the French Ministry of Health (PHRC n°20-0424). This work uses Data/Material provided by patients and collected by the NHS as part of their care and support #DataSavesLives. The Data/materials used for this research were obtained from ISARIC4C. ISARIC4C Investigators collated the COVID-19 Clinical Information Network (CO-CIN) data.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

At-admission prediction of mortality and pulmonary embolism in an international cohort of hospitalised patients with COVID-19 using statistical and machine learning methods

  • Mesinovic, Munib
  • Wong, Xin Ci
  • Rajahram, Giri Shan
  • Citarella, Barbara Wanjiru
  • Peariasamy, Kalaiarasu M.
  • van Someren Greve, Frank
  • Olliaro Piero L.
  • Merson, Laura
  • Clifton, Lei
  • Kartsonaki, Christiana
  • ISARIC Clinical Characterisation Group
By September 2022, more than 600 million cases of SARS-CoV-2 infection have been reported globally, resulting in over 6.5 million deaths. COVID-19 mortality risk estimators are often, however, developed with small unrepresentative samples and with methodological limitations. It is highly important to develop predictive tools for pulmonary embolism (PE) in COVID-19 patients as one of the most severe preventable complications of COVID-19. Early recognition can help provide life-saving targeted anti-coagulation therapy right at admission. Using a dataset of more than 800,000 COVID-19 patients from an international cohort, we propose a cost-sensitive gradient-boosted machine learning model that predicts occurrence of PE and death at admission. Logistic regression, Cox proportional hazards models, and Shapley values were used to identify key predictors for PE and death. Our prediction model had a test AUROC of 75.9% and 74.2%, and sensitivities of 67.5% and 72.7% for PE and all-cause mortality respectively on a highly diverse and held-out test set. The PE prediction model was also evaluated on patients in UK and Spain separately with test results of 74.5% AUROC, 63.5% sensitivity and 78.9% AUROC, 95.7% sensitivity. Age, sex, region of admission, comorbidities (chronic cardiac and pulmonary disease, dementia, diabetes, hypertension, cancer, obesity, smoking), and symptoms (any, confusion, chest pain, fatigue, headache, fever, muscle or joint pain, shortness of breath) were the most important clinical predictors at admission. Age, overall presence of symptoms, shortness of breath, and hypertension were found to be key predictors for PE using our extreme gradient boosted model. This analysis based on the, until now, largest global dataset for this set of problems can inform hospital prioritisation policy and guide long term clinical research and decision-making for COVID-19 patients globally. Our machine learning model developed from an international cohort can serve to better regulate hospital risk prioritisation of at-risk patients., This work was made possible by the UK Foreign, Commonwealth and Development Office and Wellcome [215091/Z/18/Z, 222410/Z/21/Z, 225288/Z/22/Z and 220757/Z/20/Z]; the Bill & Melinda Gates Foundation [OPP1209135]; the philanthropic support of the donors to the University of Oxford’s COVID-19 Research Response Fund (0009109); CIHR Coronavirus Rapid Research Funding Opportunity OV2170359 and the coordination in Canada by Sunnybrook Research Institute; endorsement of the Irish Critical Care-Clinical Trials Group, co-ordination in Ireland by the Irish Critical Care-Clinical Trials Network at University College Dublin and funding by the Health Research Board of Ireland [CTN-2014-12]; the Rapid European COVID-19 Emergency Response research (RECOVER) [H2020 project 101003589] and European Clinical Research Alliance on Infectious Diseases (ECRAID) [965313]; the COVID clinical management team, AIIMS, Rishikesh, India; the COVID-19 Clinical Management team, Manipal Hospital Whitefield, Bengaluru, India; Cambridge NIHR Biomedical Research Centre; the dedication and hard work of the Groote Schuur Hospital Covid ICU Team and supported by the Groote Schuur nursing and University of Cape Town registrar bodies coordinated by the Division of Critical Care at the University of Cape Town; the Liverpool School of Tropical Medicine and the University of Oxford; the dedication and hard work of the Norwegian SARS-CoV-2 study team and the Research Council of Norway grant no 312780, and a philanthropic donation from Vivaldi Invest A/S owned by Jon Stephenson von Tetzchner; Imperial NIHR Biomedical Research Centre; the Comprehensive Local Research Networks (CLRNs) of which PJMO is an NIHR Senior Investigator (NIHR201385); Innovative Medicines Initiative Joint Undertaking under Grant Agreement No. 115523 COMBACTE, resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007- 2013) and EFPIA companies, in-kind contribution; Stiftungsfonds zur Förderung der Bekämpfung der Tuberkulose und anderer Lungenkrankheiten of the City of Vienna, Project Number: APCOV22BGM; Italian Ministry of Health “Fondi Ricerca corrente-L1P6” to IRCCS Ospedale Sacro Cuore-Don Calabria; Australian Department of Health grant (3273191); Gender Equity Strategic Fund at University of Queensland, Artificial Intelligence for Pandemics (A14PAN) at University of Queensland, the Australian Research Council Centre of Excellence for Engineered Quantum Systems (EQUS, CE170100009), the Prince Charles Hospital Foundation, Australia; grants from Instituto de Salud Carlos III, Ministerio de Ciencia, Spain; Brazil, National Council for Scientific and Technological Development Scholarship number 303953/2018-7; the Firland Foundation, Shoreline, Washington, USA; the French COVID cohort (NCT04262921) is sponsored by INSERM and is funded by the REACTing (REsearch & ACtion emergING infectious diseases) consortium and by a grant of the French Ministry of Health (PHRC n20-0424); a grant from foundation Bevordering Onderzoek Franciscus; the South Eastern Norway Health Authority and the Research Council of Norway; Institute for Clinical Research (ICR), National Institutes of Health (NIH) supported by the Ministry of Health Malaysia; preparedness work conducted by the Short Period Incidence Study of Severe Acute Respiratory Infection; the U.S. DoD Armed Forces Health Surveillance Division, Global Emerging Infectious Diseases Branch to the U.S Naval Medical Research Unit No. TWO (NAMRU-2) (Work Unit #: P0153_21_N2). These authors would like to thank Vysnova Partners, Inc. for the management of this research project. The Lao-Oxford-Mahosot Hospital-Wellcome Trust Research Unit is funded by the Wellcome Trust.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

ISARIC-COVID-19 dataset: A Prospective, Standardized, Global Dataset of Patients Hospitalized with COVID-19

  • ISARIC Clinical Characterization Group
  • García-Gallo, Esteban
  • Merson, Laura
  • Kennon, Kalynn
  • Kelly, Sadie
  • Citarella, Barbara Wanjiru
  • Fryer, Daniel Vidali
  • Shrapnel, Sally
  • Lee, James
  • Duque, Sara
  • Fuentes, Yuli V.
  • Balan, Valeria
  • Smith, Sue
  • Wei, Jia
  • Gonçalves, Bronner P.
  • Russell, Clark D.
  • Sigfrid, Louise
  • Dagens, Andrew
  • Olliaro, Piero L.
  • Baruch, Joaquín
  • Kartsonaki, Christiana
  • Dunning, Jake
  • Rojek, Amanda
  • Rashan, Aasiyah
  • Beane, Abi
  • Murthy, Srinivas
  • Reyes, Luis Felipe
The International Severe Acute Respiratory and Emerging Infection Consortium (ISARIC) COVID-19 dataset is one of the largest international databases of prospectively collected clinical data on people hospitalized with COVID-19. This dataset was compiled during the COVID-19 pandemic by a network of hospitals that collect data using the ISARIC-World Health Organization Clinical Characterization Protocol and data tools. The database includes data from more than 705,000 patients, collected in more than 60 countries and 1,500 centres worldwide. Patient data are available from acute hospital admissions with COVID-19 and outpatient follow-ups. The data include signs and symptoms, pre-existing comorbidities, vital signs, chronic and acute treatments, complications, dates of hospitalization and discharge, mortality, viral strains, vaccination status, and other data. Here, we present the dataset characteristics, explain its architecture and how to gain access, and provide tools to facilitate its use., This work is part of the Grand Challenges ICODA pilot initiative, delivered by Health Data Research UK and funded by the Bill & Melinda Gates Foundation and the Minderoo Foundation. The philanthropic support of the donors to the University of Oxford’s COVID-19 Research Response Fund; UK Foreign, Commonwealth and Development Office and Wellcome [215091/Z/18/Z and 220757/Z/20/Z]; the Bill & Melinda Gates Foundation [OPP1209135]; the National Institute for Health Research (NIHR; award CO-CIN-01); the Medical Research Council (MRC; grant MC_PC_19059); the NIHR Health Protection Research Unit (HPRU) in Emerging and Zoonotic Infections at University of Liverpool in partnership with Public Health England (PHE)(award 200907); NIHR HPRU in Respiratory Infections at Imperial College London with PHE (award 200927); Liverpool Experimental Cancer Medicine Centre (grant C18616/A25153); NIHR Biomedical Research Centre at Imperial College London (award IS-BRC-1215-20013); NIHR Clinical Research Network (infrastructure support); CIHR Coronavirus Rapid Research Funding Opportunity OV2170359 and was coordinated out of Sunnybrook Research Institute; the endorsement of the Irish Critical Care- Clinical Trials Group, co-ordinated in Ireland by the Irish Critical Care- Clinical Trials Network at University College Dublin and funded by the Health Research Board of Ireland [CTN-2014-12]; Rapid European COVID-19 Emergency Response research (RECOVER) [H2020 project 101003589]; European Clinical Research Alliance on Infectious Diseases (ECRAID) [965313]; COVID clinical management team, AIIMS, Rishikesh, India; Cambridge NIHR Biomedical Research Centre; the dedication and hard work of the Groote Schuur Hospital Covid ICU Team; the Groote Schuur nursing and University of Cape Town registrar bodies coordinated by the Division of Critical Care at the University of Cape Town; Wellcome Trust fellowship [205228/Z/16/Z]; the Liverpool School of Tropical Medicine; the University of Oxford; the dedication and hard work of the Norwegian SARS-CoV-2 study team; the Research Council of Norway grant no 312780; a philanthropic donation from Vivaldi Invest A/S owned by Jon Stephenson von Tetzchner; Innovative Medicines Initiative Joint Undertaking under Grant Agreement No. 115523 COMBACTE, resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007–2013) and EFPIA companies, in-kind contribution; preparedness work conducted by the Short Period Incidence Study of Severe Acute Respiratory Infection; Stiftungsfonds zur Förderung der Bekämpfung der Tuberkulose und anderer Lungenkrankheiten of the City of Vienna, Project Number: APCOV22BGM; Italian Ministry of Health “Fondi Ricerca corrente–L1P6” to IRCCS Ospedale Sacro Cuore–Don Calabria; Australian Department of Health grant (3273191); Gender Equity Strategic Fund at University of Queensland; Artificial Intelligence for Pandemics (A14PAN) at University of Queensland; the Australian Research Council Centre of Excellence for Engineered Quantum Systems (EQUS, CE170100009); the Prince Charles Hospital Foundation, Australia; UK Medical Research Council Clinical Research Training Fellowship MR/V001671/1; Instituto de Salud Carlos III, Ministerio de Ciencia, Spain; Brazil, National Council for Scientific and Technological Development Scholarship number 303953/2018-7; Firland Foundation, Shoreline, Washington, USA; the French COVID cohort (NCT04262921) is sponsored by INSERM and is funding by the REACTing (REsearch & ACtion emergING infectious diseases) consortium and by a grant of the French Ministry of Health (PHRC n°20-0424)., Peer reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Lightweight transformers for clinical natural language processing

  • Rohanian, Omid
  • Nouriborji, Mohammadmahdi
  • Jauncey, Hannah
  • Kouchaki, Samaneh
  • Nooralahzadeh, Farhad
  • ISARIC Clinical Characterisation Group
  • Clifton, Lei
  • Merson, Laura
  • Clifton, David A.
Specialised pre-trained language models are becoming more frequent in Natural language Processing (NLP) since they can potentially outperform models trained on generic texts. BioBERT (Sanh et al., Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv: 1910.01108, 2019) and BioClinicalBERT (Alsentzer et al., Publicly available clinical bert embeddings. In Proceedings of the 2nd Clinical Natural Language Processing Workshop, pp. 72-78, 2019) are two examples of such models that have shown promise in medical NLP tasks. Many of these models are overparametrised and resource-intensive, but thanks to techniques like knowledge distillation, it is possible to create smaller versions that perform almost as well as their larger counterparts. In this work, we specifically focus on development of compact language models for processing clinical texts (i.e. progress notes, discharge summaries, etc). We developed a number of efficient lightweight clinical transformers using knowledge distillation and continual learning, with the number of parameters ranging from million to million. These models performed comparably to larger models such as BioBERT and ClinicalBioBERT and significantly outperformed other compact models trained on general or biomedical data. Our extensive evaluation was done across several standard datasets and covered a wide range of clinical text-mining tasks, including natural language inference, relation extraction, named entity recognition and sequence classification. To our knowledge, this is the first comprehensive study specifically focused on creating efficient and compact transformers for clinical NLP tasks. The models and code used in this study can be found on our Huggingface profile at https://huggingface.co/nlpie and Github page at https://github.com/nlpie-research/Lightweight-Clinical-Transformers, respectively, promoting reproducibility of our results., This work was made possible with the support of UK Foreign, Commonwealth and Development Office and Wellcome [225288/Z/22/Z]. Collection of data for the ISARIC Clinical Notes was made possible with the support of UK Foreign, Commonwealth and Development Office and Wellcome [215091/Z/18/Z, 222410/Z/21/Z, 225288/Z/22/Z, 220757/Z/20/Z and 222048/Z/20/Z] and the Bill & Melinda Gates Foundation [OPP1209135]; CIHR Coronavirus Rapid Research Funding Opportunity OV2170359 and was coordinated out of Sunnybrook Research Institute; was supported by endorsement of the Irish Critical Care- Clinical Trials Group, co-ordinated in Ireland by the Irish Critical Care- Clinical Trials Network at University College Dublin and funded by the Health Research Board of Ireland [CTN-2014-12]; grants from Rapid European COVID-19 Emergency Response research (RECOVER) [H2020 project 101003589] and European Clinical Research Alliance on Infectious Diseases (ECRAID) [965313]; Cambridge NIHR Biomedical Research Centre; Wellcome Trust fellowship [205228/Z/16/Z] and the National Institute for Health Research Health Protection Research Unit (HPRU) in Emerging and Zoonotic Infections (NIHR200907) at the University of Liverpool in partnership with Public Health England (PHE), in collaboration with Liverpool School of Tropical Medicine and the University of Oxford; the dedication and hard work of the Norwegian SARS-CoV-2 study team. Research Council of Norway grant no 312780 and a philanthropic donation from Vivaldi Invest A/S owned by Jon Stephenson von Tetzchner; PJMO was supported by the UK’s National Institute for Health Research (NIHR) via Imperial’s Biomedical Research Centre (NIHR Imperial BRC), Imperial’s Health Protection Research Unit in Respiratory Infections (NIHR HPRU RI), the Comprehensive Local Research Networks (CLRNs) and is an NIHR Senior Investigator (NIHR201385); Innovative Medicines Initiative Joint Undertaking under Grant Agreement No. 115523 COMBACTE, resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies, in-kind contribution; Stiftungsfonds zur Förderung der Bekämpfung der Tuberkulose und anderer Lungenkrankheiten of the City of Vienna; Project Number: APCOV22BGM; Australian Department of Health grant (3273191); Gender Equity Strategic Fund at University of Queensland, Artificial Intelligence for Pandemics (A14PAN) at University of Queensland, The Australian Research Council Centre of Excellence for Engineered Quantum Systems (EQUS, CE170100009), The Prince Charles Hospital Foundation, Australia; grants from Instituto de Salud Carlos III, Ministerio de Ciencia, Spain; Brazil, National Council for Scientific and Technological Development Scholarship number 303953/2018-7; the Firland Foundation, Shoreline, Washington, USA; The French COVID cohort (NCT04262921) is sponsored by INSERM and is funding by the REACTing (REsearch & ACtion emergING infectious diseases) consortium and by a grant of the French Ministry of Health (PHRC n 20-0424); the South Eastern Norway Health Authority and the Research Council of Norway; and a grant from the Oxford University COVID-19 Research Response fund (grant 0009109); Institute for Clinical Research (ICR), National Institutes of Health (NIH) supported by the Ministry of Health Malaysia; a grant from foundation Bevordering Onderzoek Franciscus.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

ISARIC-COVID-19 dataset: A prospective, standardized, global dataset of patients hospitalized with COVID-19

  • ISARIC Clinical Characterisation Group
  • Reyes, Luis F.
The International Severe Acute Respiratory and Emerging Infection Consortium (ISARIC) COVID-19 dataset is one of the largest international databases of prospectively collected clinical data on people hospitalized with COVID-19. This dataset was compiled during the COVID-19 pandemic by a network of hospitals that collect data using the ISARIC-World Health Organization Clinical Characterization Protocol and data tools. The database includes data from more than 705,000 patients, collected in more than 60 countries and 1,500 centres worldwide. Patient data are available from acute hospital admissions with COVID-19 and outpatient follow-ups. The data include signs and symptoms, pre-existing comorbidities, vital signs, chronic and acute treatments, complications, dates of hospitalization and discharge, mortality, viral strains, vaccination status, and other data. Here, we present the dataset characteristics, explain its architecture and how to gain access, and provide tools to facilitate its use., This work is part of the Grand Challenges ICODA pilot initiative, delivered by Health Data Research UK and funded by the Bill & Melinda Gates Foundation and the Minderoo Foundation. The philanthropic support of the donors to the University of Oxford’s COVID-19 Research Response Fund; UK Foreign, Commonwealth and Development Office and Wellcome [215091/Z/18/Z and 220757/Z/20/Z]; the Bill & Melinda Gates Foundation [OPP1209135]; the National Institute for Health Research (NIHR; award CO-CIN-01); the Medical Research Council (MRC; grant MC_PC_19059); the NIHR Health Protection Research Unit (HPRU) in Emerging and Zoonotic Infections at University of Liverpool in partnership with Public Health England (PHE)(award 200907); NIHR HPRU in Respiratory Infections at Imperial College London with PHE (award 200927); Liverpool Experimental Cancer Medicine Centre (grant C18616/A25153); NIHR Biomedical Research Centre at Imperial College London (award IS-BRC-1215-20013); NIHR Clinical Research Network (infrastructure support); CIHR Coronavirus Rapid Research Funding Opportunity OV2170359 and was coordinated out of Sunnybrook Research Institute; the endorsement of the Irish Critical Care- Clinical Trials Group, co-ordinated in Ireland by the Irish Critical Care- Clinical Trials Network at University College Dublin and funded by the Health Research Board of Ireland [CTN-2014-12]; Rapid European COVID-19 Emergency Response research (RECOVER) [H2020 project 101003589]; European Clinical Research Alliance on Infectious Diseases (ECRAID) [965313]; COVID clinical management team, AIIMS, Rishikesh, India; Cambridge NIHR Biomedical Research Centre; the dedication and hard work of the Groote Schuur Hospital Covid ICU Team; the Groote Schuur nursing and University of Cape Town registrar bodies coordinated by the Division of Critical Care at the University of Cape Town; Wellcome Trust fellowship [205228/Z/16/Z]; the Liverpool School of Tropical Medicine; the University of Oxford; the dedication and hard work of the Norwegian SARS-CoV-2 study team; the Research Council of Norway grant no 312780; a philanthropic donation from Vivaldi Invest A/S owned by Jon Stephenson von Tetzchner; Innovative Medicines Initiative Joint Undertaking under Grant Agreement No. 115523 COMBACTE, resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007–2013) and EFPIA companies, in-kind contribution; preparedness work conducted by the Short Period Incidence Study of Severe Acute Respiratory Infection; Stiftungsfonds zur Förderung der Bekämpfung der Tuberkulose und anderer Lungenkrankheiten of the City of Vienna, Project Number: APCOV22BGM; Italian Ministry of Health “Fondi Ricerca corrente–L1P6” to IRCCS Ospedale Sacro Cuore–Don Calabria; Australian Department of Health grant (3273191); Gender Equity Strategic Fund at University of Queensland; Artificial Intelligence for Pandemics (A14PAN) at University of Queensland; the Australian Research Council Centre of Excellence for Engineered Quantum Systems (EQUS, CE170100009); the Prince Charles Hospital Foundation, Australia; UK Medical Research Council Clinical Research Training Fellowship MR/V001671/1; Instituto de Salud Carlos III, Ministerio de Ciencia, Spain; Brazil, National Council for Scientific and Technological Development Scholarship number 303953/2018-7; Firland Foundation, Shoreline, Washington, USA; the French COVID cohort (NCT04262921) is sponsored by INSERM and is funding by the REACTing (REsearch & ACtion emergING infectious diseases) consortium and by a grant of the French Ministry of Health (PHRC n°20-0424). This work uses Data/Material provided by patients and collected by the NHS as part of their care and support #DataSavesLives. The Data/materials used for this research were obtained from ISARIC4C. ISARIC4C Investigators collated the COVID-19 Clinical Information Network (CO-CIN) data.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

At-admission prediction of mortality and pulmonary embolism in an international cohort of hospitalised patients with COVID-19 using statistical and machine learning methods

  • Mesinovic, Munib
  • Wong, Xin Ci
  • Rajahram, Giri Shan
  • Citarella, Barbara Wanjiru
  • Peariasamy, Kalaiarasu M.
  • van Someren Greve, Frank
  • Olliaro Piero L.
  • Merson, Laura
  • Clifton, Lei
  • Kartsonaki, Christiana
  • ISARIC Clinical Characterisation Group
By September 2022, more than 600 million cases of SARS-CoV-2 infection have been reported globally, resulting in over 6.5 million deaths. COVID-19 mortality risk estimators are often, however, developed with small unrepresentative samples and with methodological limitations. It is highly important to develop predictive tools for pulmonary embolism (PE) in COVID-19 patients as one of the most severe preventable complications of COVID-19. Early recognition can help provide life-saving targeted anti-coagulation therapy right at admission. Using a dataset of more than 800,000 COVID-19 patients from an international cohort, we propose a cost-sensitive gradient-boosted machine learning model that predicts occurrence of PE and death at admission. Logistic regression, Cox proportional hazards models, and Shapley values were used to identify key predictors for PE and death. Our prediction model had a test AUROC of 75.9% and 74.2%, and sensitivities of 67.5% and 72.7% for PE and all-cause mortality respectively on a highly diverse and held-out test set. The PE prediction model was also evaluated on patients in UK and Spain separately with test results of 74.5% AUROC, 63.5% sensitivity and 78.9% AUROC, 95.7% sensitivity. Age, sex, region of admission, comorbidities (chronic cardiac and pulmonary disease, dementia, diabetes, hypertension, cancer, obesity, smoking), and symptoms (any, confusion, chest pain, fatigue, headache, fever, muscle or joint pain, shortness of breath) were the most important clinical predictors at admission. Age, overall presence of symptoms, shortness of breath, and hypertension were found to be key predictors for PE using our extreme gradient boosted model. This analysis based on the, until now, largest global dataset for this set of problems can inform hospital prioritisation policy and guide long term clinical research and decision-making for COVID-19 patients globally. Our machine learning model developed from an international cohort can serve to better regulate hospital risk prioritisation of at-risk patients., This work was made possible by the UK Foreign, Commonwealth and Development Office and Wellcome [215091/Z/18/Z, 222410/Z/21/Z, 225288/Z/22/Z and 220757/Z/20/Z]; the Bill & Melinda Gates Foundation [OPP1209135]; the philanthropic support of the donors to the University of Oxford’s COVID-19 Research Response Fund (0009109); CIHR Coronavirus Rapid Research Funding Opportunity OV2170359 and the coordination in Canada by Sunnybrook Research Institute; endorsement of the Irish Critical Care-Clinical Trials Group, co-ordination in Ireland by the Irish Critical Care-Clinical Trials Network at University College Dublin and funding by the Health Research Board of Ireland [CTN-2014-12]; the Rapid European COVID-19 Emergency Response research (RECOVER) [H2020 project 101003589] and European Clinical Research Alliance on Infectious Diseases (ECRAID) [965313]; the COVID clinical management team, AIIMS, Rishikesh, India; the COVID-19 Clinical Management team, Manipal Hospital Whitefield, Bengaluru, India; Cambridge NIHR Biomedical Research Centre; the dedication and hard work of the Groote Schuur Hospital Covid ICU Team and supported by the Groote Schuur nursing and University of Cape Town registrar bodies coordinated by the Division of Critical Care at the University of Cape Town; the Liverpool School of Tropical Medicine and the University of Oxford; the dedication and hard work of the Norwegian SARS-CoV-2 study team and the Research Council of Norway grant no 312780, and a philanthropic donation from Vivaldi Invest A/S owned by Jon Stephenson von Tetzchner; Imperial NIHR Biomedical Research Centre; the Comprehensive Local Research Networks (CLRNs) of which PJMO is an NIHR Senior Investigator (NIHR201385); Innovative Medicines Initiative Joint Undertaking under Grant Agreement No. 115523 COMBACTE, resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007- 2013) and EFPIA companies, in-kind contribution; Stiftungsfonds zur Förderung der Bekämpfung der Tuberkulose und anderer Lungenkrankheiten of the City of Vienna, Project Number: APCOV22BGM; Italian Ministry of Health “Fondi Ricerca corrente-L1P6” to IRCCS Ospedale Sacro Cuore-Don Calabria; Australian Department of Health grant (3273191); Gender Equity Strategic Fund at University of Queensland, Artificial Intelligence for Pandemics (A14PAN) at University of Queensland, the Australian Research Council Centre of Excellence for Engineered Quantum Systems (EQUS, CE170100009), the Prince Charles Hospital Foundation, Australia; grants from Instituto de Salud Carlos III, Ministerio de Ciencia, Spain; Brazil, National Council for Scientific and Technological Development Scholarship number 303953/2018-7; the Firland Foundation, Shoreline, Washington, USA; the French COVID cohort (NCT04262921) is sponsored by INSERM and is funded by the REACTing (REsearch & ACtion emergING infectious diseases) consortium and by a grant of the French Ministry of Health (PHRC n20-0424); a grant from foundation Bevordering Onderzoek Franciscus; the South Eastern Norway Health Authority and the Research Council of Norway; Institute for Clinical Research (ICR), National Institutes of Health (NIH) supported by the Ministry of Health Malaysia; preparedness work conducted by the Short Period Incidence Study of Severe Acute Respiratory Infection; the U.S. DoD Armed Forces Health Surveillance Division, Global Emerging Infectious Diseases Branch to the U.S Naval Medical Research Unit No. TWO (NAMRU-2) (Work Unit #: P0153_21_N2). These authors would like to thank Vysnova Partners, Inc. for the management of this research project. The Lao-Oxford-Mahosot Hospital-Wellcome Trust Research Unit is funded by the Wellcome Trust.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

A multi-country analysis of COVID-19 hospitalizations by vaccination status

  • Gonçalves, Bronner P.
  • Martin-Loeches, Ignacio
  • Merson, Laura
  • ISARIC Clinical Characterisation Group
Background. Individuals vaccinated against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), when infected, can still develop disease that requires hospitalization. It remains unclear whether these patients differ from hospitalized unvaccinated patients with regard to presentation, coexisting comorbidities, and outcomes. Methods. Here, we use data from an international consortium to study this question and assess whether differences between these groups are context specific. Data from 83,163 hospitalized COVID-19 patients (34,843 vaccinated, 48,320 unvaccinated) from 38 countries were analyzed. Findings. While typical symptoms were more often reported in unvaccinated patients, comorbidities, including some associated with worse prognosis in previous studies, were more common in vaccinated patients. Considerable between-country variation in both in-hospital fatality risk and vaccinated-versus-unvaccinated difference in this outcome was observed. Conclusions. These findings will inform allocation of healthcare resources in future surges as well as design of longer-term international studies to characterize changes in clinical profile of hospitalized COVID-19 patients related to vaccination history., This work was also funded by grants from the National Institute for Health Research (NIHR; award CO-CIN-01), the Medical Research Council (MRC; grant MC_PC_19059), and the NIHR Health Protection Research Unit (HPRU) in Emerging and Zoonotic Infections at the University of Liverpool in partnership with Public Health England (PHE) (award 200907), NIHR HPRU in Respiratory Infections at Imperial College London with PHE (award 200927), Liverpool Experimental Cancer Medicine Centre (grant C18616/A25153), NIHR Biomedical Research Centre at Imperial College London (award ISBRC-1215-20013), and NIHR Clinical Research Network providing infrastructure support; CIHR Coronavirus Rapid Research Funding Opportunity OV2170359 and the coordination in Canada by Sunnybrook Research Institute; funding by the Health Research Board of Ireland (CTN-2014-12); the Rapid European COVID-19 Emergency Response research (RECOVER) (H2020 project 101003589) and European Clinical Research Alliance on Infectious Diseases (ECRAID) (965313); Cambridge NIHR Biomedical Research Centre (award NIHR203312); a Research Council of Norway grant no. 312780 and a philanthropic donation from Vivaldi Invest A/S owned by Jon Stephenson von Tetzchner; the Comprehensive Local Research Networks (CLRNs), of which P.J.M.O. is an NIHR Senior Investigator (NIHR201385); Stiftungsfonds zur Förderung der Bekämpfung der Tuberkulose und anderer Lungenkrankheiten of the City of Vienna, project number APCOV22BGM; funding from Medical University of Vienna, Department of Anaesthesia, Intensive Care Medicine and Pain Medicine; Italian Ministry of Health “Fondi Ricerca corrente–L1P6” to IRCCS Ospedale Sacro Cuore–Don Calabria; Australian Department of Health grant (3273191); Gender Equity Strategic Fund at University of Queensland, Artificial Intelligence for Pandemics (A14PAN) at University of Queensland, the Australian Research Council Centre of Excellence for Engineered Quantum Systems (EQUS, CE170100009), the Prince Charles Hospital Foundation, Australia; a grant from foundation Bevordering Onderzoek Franciscus; the South Eastern Norway Health Authority and the Research Council of Norway; Institute of Cancer Research (ICR), National Institutes of Health (NIH) supported by the Ministry of Health Malaysia; and the US DoD Armed Forces Health Surveillance Division, Global Emerging Infectious Diseases Branch to the US Naval Medical ResearchUnit no. TWO (NAMRU-2) (Work Unit #: P0153_21_N2). These authors would like to thank Vysnova Partners, Inc., for the management of this research project. The Lao-Oxford-Mahosot Hospital-Wellcome Trust Research Unit is funded by the Wellcome Trust; M.U.G.K. was supported by the Branco Weiss Fellowship, Google.org, the Oxford Martin School, the Rockefeller Foundation, and the European Union Horizon 2020 project MOOD (#874850). The contents of this publication are the sole responsibility of the authors and do not necessarily reflect the views of the European Commission. The investigators acknowledge the support of the COVID clinical management team, AIIMS, Rishikesh, India; the COVID-19 Clinical Management team, Manipal Hospital Whitefield, Bengaluru, India; the Liverpool School of Tropical Medicine and the University of Oxford; Imperial NIHR Biomedical Research Centre; the dedication and hard work of the Norwegian SARS-CoV-2 study team; endorsement of the Irish Critical Care Clinical Trials Group, co-ordination in Ireland by the Irish Critical Care Clinical Trials Network at University College Dublin; and preparedness work conducted by the Short Period Incidence Study of Severe Acute Respiratory Infection. This work uses data provided by patients and collected by the NHS as part of their care and support #DataSavesLives. The data used for this research were obtained from ISARIC4C. We are extremely grateful to the 2,648 frontline NHS clinical and research staff and volunteer medical students who collected these data in challenging circumstances and for the generosity of the patients and their families for their individual contributions in these difficult times. The COVID-19 Clinical Information Network (CO-CIN) data were collated by ISARIC4C Investigators. We also acknowledge the support of Jeremy J. Farrar and Nahoko Shindo.