Skip to main content

MODELIZACION DE LA CINEMATICA DE LA INFORMACION PARA UNA INTELIGENCIA ARTIFICIAL MEDICA RESISTENTE A LOS CAMBIOS

PID2022-138636OA-I00

•
Funding agency name Agencia Estatal de Investigación
Funding agency acronym AEI
Program Programa Estatal para Impulsar la Investigación Científico-Técnica y su Transferencia
Subprogram Subprograma Estatal de Generación de Conocimiento
Call Proyectos de I+D+i (Generación de Conocimiento y Retos Investigación)
Call year 2022
State Plan 2021-2023 Plan Estatal de Investigación Científica y Técnica y de Innovación
Beneficiary centre UNIVERSITAT POLITÈCNICA DE VALÈNCIA

Resultados relacionados

Resultados totales (Incluyendo duplicados): 5
Encontrada(s) 1 página(s)

RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

Unsupervised Characterization of Temporal Dataset Shifts as an Early Indicator of AI Performance Variations: Evaluation Study Using the Medical Information Mart for Intensive Care-IV Dataset

  • Fernandez-Narro, David
  • Ferri-Borredà, Pablo|||0000-0002-3083-1169
  • Garcia-Gomez, Juan M
  • Sáez Silvestre, Carlos|||0000-0003-2678-8249
  • Gutiérrez-Sacristán, Alba
[EN] Background: Reusing long-term data from electronic health records is essential for training reliable and effective health artificial intelligence (AI). However, intrinsic changes in health data distributions over time-known as dataset shifts, which include concept, covariate, and prior shifts-can compromise model performance, leading to model obsolescence and inaccurate decisions. Objective: In this study, we investigate whether unsupervised, model-agnostic characterization of temporal dataset shifts using data distribution analyses through Information Geometric Temporal (IGT) projections is an early indicator of potential AI performance variations before model development. Methods: Using the real-world Medical Information Mart for Intensive Care-IV (MIMIC-IV) electronic health record database, encompassing data from over 40,000 patients from 2008 to 2019, we characterized its inherent dataset shift patterns through an unsupervised approach using IGT projections and data temporal heatmaps. We trained and evaluated annually a set of random forests and gradient boosting models to predict in-hospital mortality. To assess the impact of shifts on model performance, we checked the association between the temporal clusters found in both IGT projections and the intertime embedding of model performances using the Fisher exact test. Results: Our results demonstrate a significant relationship between the unsupervised temporal shift patterns, specifically covariate and concept shifts, identified using the IGT projection method and the performance of the random forest and gradient boosting models (P<.05). We identified 2 primary temporal clusters that correspond to the periods before and after ICD-10 (International Statistical Classification of Diseases, Tenth Revision) implementation. The transition from ICD-9 (International Classification of Diseases, Ninth Revision) to ICD-10 was a major source of dataset shift, associated with a performance degradation. Conclusions: Unsupervised, model-agnostic characterization of temporal shifts via IGT projections can serve as a proactive monitoring tool to anticipate performance shifts in clinical AI models. By incorporating early shift detection into the development pipeline, we can enhance decision-making during the training and maintenance of these models. This approach paves the way for more robust, trustworthy, and self-adapting AI systems in health care., This study was funded by the project PID2022-138636OA-I00 (KINEMAI) by Agencia Estatal de Investigacion-Proyectos de Generacion de Conocimiento 2022.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

Acute brain dysfunction clusters in COVID-19: a pilot machine learning-based analysis of the COVID-D cohort

  • Romero-Garcia, Nekane
  • Hurtado, Luis
  • Passariello, Eduardo
  • Taccone, Fabio Silvio
  • Robba, Chiara
  • Raman, Rameela
  • Orun, Onur M.
  • Pandharipande, Pratik
  • Pun, Brenda T.
  • Ely, E. Wesley
  • Badenes, Rafael
  • Montosa-Micó, Víctor
  • Fernandez-Narro, David
  • Garcia-Gomez, Juan M
  • Sáez Silvestre, Carlos|||0000-0003-2678-8249
[EN] Purpose While acute brain dysfunction (ABD, i.e., delirium and coma) is associated with significantly increased morbidity in critically ill patients, it presents with great heterogeneity that poses a challenge for management and prognostication. While machine learning may be promising for subgroup identification, this approach has not yet been applied to COVID-19 patients with ABD. The aim of our study was to identify distinct clusters among critically ill patients with COVID-19 based on ICU admission data and evaluate their association with clinical outcomes. Methods We retrospectively analyzed an international multicenter database (COVID-D study) of critically ill adult patients with COVID-19 during the first pandemic wave and ABD using clinical features on day 1 of admission as input variables. We applied unsupervised machine learning in a pilot attempt to discover clusters of ABD patients. Hierarchical clustering was performed with a bootstrap-based robustness assessment after dimensionality reduction. Clusters were analyzed for differences in neurological outcomes, mechanical ventilation, and survival. Results We analyzed 1,631 critically ill COVID-19 patients with ABD, identifying four reproducible clusters with distinct clinical and neurological profiles. Cluster 1 ("mild respiratory failure," n = 335) had the most favorable outcomes, with the shortest duration of delirium (4.13 days) and mechanical ventilation. Cluster 2 ("moderate ARDS," n = 508) showed a comparable delirium incidence but the longest duration (5.18 days). Cluster 3 ("early severe ARDS," n = 161) included patients who underwent prone positioning and mechanical ventilation early from the day of admission, with higher rates of coma (100%), including persistent coma (27.3%). Cluster 4 ("late severe ARDS," n = 475) represented severely ill patients with the longest coma duration (11.2 days) and the lowest delirium-free and coma-free (DFCF) days (4.74), in relation to deep and prolonged sedation. Despite the wide range of ABD durations across four groups, no significantly different 28-day mortality (23.6-38.0, p > 0.78), ICU (15.8-19.2 days range, p = 0.154) and hospital (22.5-26.7 days range, p = 0.259) length of stay were observed among clusters. Conclusion This pilot analysis of ICU admission data from the first COVID-19 wave suggests the existence of clinically distinct clusters among patients with acute brain dysfunction. Differences were observed in the type and duration of delirium and coma, though these did not translate into differences in 28-day survival. This exploratory work may support targeted delirium prevention strategies, but prospective studies are required to determine its clinical utility in modern ICU settings., This work was funded by Programa INVESTIGO (INVEST/2022/298) by Generalitat Valenciana; Modelizacion de la Cinematica de la Informacion para una Inteligencia Artificial Medica Resistente a los Cambios (KINEMAI) (PID2022-138636OA-I00).
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

An end-to-end solution for out-of-hospital emergency medical dispatch triage based on multimodal and continual deep learning

  • Ferri-Borredà, Pablo|||0000-0002-3083-1169
  • Sáez Silvestre, Carlos|||0000-0003-2678-8249
  • Garcia-Gomez, Juan M
  • Félix-De Castro, Antonio
  • Sánchez-Cuesta, Purificación
[EN] The objective of this study was to build a multimodal, multitask predictive model-named E2eDeepEMC2-to improve out-of-hospital emergency incident severity assessments while coping with shifts in data distributions over time. We drew on 2054694 independent incidents recorded by the Valencian emergency medical dispatch service between 2009 and 2019 (excluding 2013), combining demographic, temporal, clinical and free-text inputs. To handle temporal drift, our model integrates continual learning strategies and comprises three encoder modules (for context, clinical data and text), whose outputs are merged to predict the life-threatening level, admissible response delay and emergency system jurisdiction. Compared with the Valencian Region's existing in-house triage protocol, E2eDeepEMC2 achieved absolute F1-score gains of 18.46% for life-threatening level, 25.96% for response delay and 3.63% for jurisdiction. Compared to non-continual learning baselines, it also outperformed them by 3.04%, 9.66% and 0.58%, respectively. Deployment of E2eDeepEMC2 is currently underway in the Valencian Region, underscoring its practical impact on real-world emergency dispatch decision-making., This work has received support from the Ministry of Science, Innovation, and Universities of Spain through the FPU18/06441 program and the KINEMAI project (PID2022-138636OA-I00) .
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

Resilient Artificial Intelligence in Health: Synthesis and Research Agenda Toward Next-Generation Trustworthy Clinical Decision Support

  • Sáez Silvestre, Carlos|||0000-0003-2678-8249
  • Ferri-Borredà, Pablo|||0000-0002-3083-1169
  • Garcia-Gomez, Juan M
[EN] Artificial intelligence (AI)-based clinical decision support systems are gaining momentum by relying on a greater volume and variety of secondary use data. However, the uncertainty, variability, and biases in real -world data environments still pose significant challenges to the development of health AI, its routine clinical use, and its regulatory frameworks. Health AI should be resilient against real -world environments throughout its lifecycle, including the training and prediction phases and maintenance during production, and health AI regulations should evolve accordingly. Data quality issues, variability over time or across sites, information uncertainty, human -computer interaction, and fundamental rights assurance are among the most relevant challenges. If health AI is not designed resiliently with regard to these real -world data effects, potentially biased data -driven medical decisions can risk the safety and fundamental rights of millions of people. In this viewpoint, we review the challenges, requirements, and methods for resilient AI in health and provide a research framework to improve the trustworthiness of next -generation AI -based clinical decision support., This work was cofunded (PID2022-138636OA-I00; KINEMAI) by Agencia Estatal de Investigacion-Proyectos de Generacion de Conocimiento 2022.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

Deep continual multitask out-of-hospital incident severity assessment from changing clinical features

  • Ferri-Borredà, Pablo|||0000-0002-3083-1169
  • Sáez Silvestre, Carlos|||0000-0003-2678-8249
  • Garcia-Gomez, Juan M
  • Antonio Félix-De Castro
  • Purificación Sánchez-Cuesta
[EN] Purpose: When developing machine learning models to support emergency medical triage, it is important to con¿ sider how changes over time in the input features can negatively affect the models¿ performance. The objective of this study was to assess the effectiveness of novel deep continual learning pipelines in maximizing model performance when input features change over time, including the emergence of new features and the disappearance of existing ones. Methods: The model is designed to identify life-threatening situations, predict their admissible response delay, and determine their institutional jurisdiction. We analyzed a total of 1 414 575 events spanning from 2009 to 2019. We provided empirical evidence for covariate shifts and measured their negative effects on model performance. Then, we proposed a set of continual learning strategies to deal with 1) changing feature domains and 2) parameter updating over time. For changing feature domains, we proposed and assessed a static domain approach, a dynamic domain approach, and a predefined approach. For parameter updating over time, we designed and evaluated five approaches: from-scratch, fine-tuning, a cumulative, rehearsal and Elastic Weight Consolidation (EWC). Results: Our findings demonstrate performance improvements, with the best strategy combination¿dynamic fea¿ ture domain coupled with EWC¿yielding gains of up to 5.9% in life-threatening situations, 18.6% in response delay, and 2.4% in jurisdiction, in absolute F1-score compared to the current triage protocol. Additionally, we observed improvements of up to 5.4% in life-threatening situations and 11% in response delay, in absolute F1-score, compared to non-continual approaches. Conclusion: The methods proposed in this work improve performance by mitigating the negative effects of covari¿ ate shifts., This work has received support from the Ministry of Science, Innovation, and Universities of Spain through the FPU18/06441 program and the KINEMAI project (PID2022-138636OA-I00).