Pasar al contenido principal

METODOS DE VISION ARTIFICIAL EN ENTORNOS DE POCAS ANOTACIONES: ANALISIS DE IMAGEN BIOMEDICA Y FACIAL

PID2021-126701OB-I00

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal para Impulsar la Investigación Científico-Técnica y su Transferencia
Subprograma Subprograma Estatal de Generación de Conocimiento
Convocatoria Proyectos de I+D+i (Generación de Conocimiento y Retos Investigación)
Año convocatoria 2021
Plan Estatal 2021-2023 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario UNIVERSIDAD DEL PAIS VASCO EUSKAL HERRIKO UNIBERTSITATEA

Resultados relacionados

Resultados totales (Incluyendo duplicados): 21
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

A deep learning method that identifies cellular heterogeneity using nanoscale nuclear features

  • Carnevali, Davide
  • Zhong, Limei
  • González-Almela, Esther
  • Viana, Carlotta
  • Rotkevich, Mikhail
  • Wang, Aiping
  • Franco-Barranco, Daniel
  • González-Marfil, Aitor
  • Neguembor, Maria Victoria
  • Castells García, Àlvaro, 1991-
  • Arganda-Carreras, Ignacio
  • Cosma, Maria Pia
Cellular phenotypic heterogeneity is an important hallmark of many biological processes and understanding its origins remains a substantial challenge. This heterogeneity often reflects variations in the chromatin structure, influenced by factors such as viral infections and cancer, which dramatically reshape the cellular landscape. To address the challenge of identifying distinct cell states, we developed artificial intelligence of the nucleus (AINU), a deep learning method that can identify specific nuclear signatures at the nanoscale resolution. AINU can distinguish different cell states based on the spatial arrangement of core histone H3, RNA polymerase II or DNA from super-resolution microscopy images. With only a small number of images as the training data, AINU correctly identifies human somatic cells, human-induced pluripotent stem cells, very early stage infected cells transduced with DNA herpes simplex virus type 1 and even cancer cells after appropriate retraining. Finally, using AI interpretability methods, we find that the RNA polymerase II localizations in the nucleoli aid in distinguishing human-induced pluripotent stem cells from their somatic cells. Overall, AINU coupled with super-resolution microscopy of nuclear structures provides a robust tool for the precise detection of cellular heterogeneity, with considerable potential for advancing diagnostics and therapies in regenerative medicine, virology and cancer biology., We acknowledge support from the European Union’s Horizon 2020 Research and Innovation Programme for two projects (no. 964342 and no. 686637 to M.P.C.); Ministerio de Ciencia e Innovación (grant no. PID2020-114080GB-I00/AEI/10.13039/501100011033 and grant no. BFU2017-86760-P/AEI/FEDER, UE to M.P.C.) and an AGAUR grant from the Departament de Recerca i Universitats de la Generalitat de Catalunya (2021-SGR2021-01300 to M.P.C.); the People Program (Marie Curie Actions) FP7/2007–2013 under REA (grant no. 608959 to M.V.N.); Ministerio de Economia y Competitvidad’s Juan de la Cierva-Incorporación 2017 (IJCI-2017-31831 to M.V.N.); Institucio Catalana de Recerca i Estudis Avançats (ICREA) (to M.P.C.); Ministerio Ciencia e Innovación and ‘ERDF A way of making Europe’ (AEI/10.13039/501100011033 grant PID2021-126701OB-I00 to I.A.-C.); the National Natural Science Foundation of China (31971177 and 32270577 to M.P.C. and 32250410296 to A.C.-G.); and Guangzhou Key Projects of Brain Science and Brain-Like Intelligence Technology (20200730009 to M.P.C.). We acknowledge support of the Spanish Ministry of Science and Innovation through the Centro de Excelencia Severo Ochoa (CEX2020-001049-S, MCIN/AEI/10.13039/501100011033), and the Generalitat de Catalunya through the CERCA programme. We are grateful to the CRG Core Technologies Programme for their support and assistance in this work. We acknowledge D. Pei (GIBH, China) for kindly sharing the urine epithelial cell line; Z. Li and G. Pan (GIBH, China) for kindly sharing the urine-epithelial-cell-derived iPSC; M. A. Sanz and L. Carrasco (Centro de Biologia Molecular Severo Ochoa, CBMSO, Spain) for kindly sharing HSV-1 (KOS strain); and B. Huang (UCSF, USA) for kindly sharing Insight3 software.
RUA. Repositorio Institucional de la Universidad de Alicante
Publicación Otros (Other)

Automatic mapping of aquaculture activity in the Atlantic Ocean

  • Lekunberri, Xabier
  • Ballester-Berman, J. David
  • Arganda-Carreras, Ignacio
  • Fernandes-Salvador, Jose A.
  • Ballester-Berman, J. David|||/items/a6960cca-2ad9-4d2b-ab77-465bf799fba8
The production of wild fish has remained relatively stable in the last two decades, whereas aquaculture organism production has increased to the point where it has exceeded wild catches. In this context, accurate and up-to-date information about the current usage of marine areas for aquaculture is crucial for the planning of marine activities. However, this data is often limited to national authorities, and discrepancies between planned and real practices can arise in available data. In this study, a novel methodology to automatically map and verify the current activity of aquaculture crops across European regions based on freely available satellite data is proposed. The European Space Agency’s (ESA) Sentinel-1 mission provides Synthetic Aperture Radar (SAR) images, which serve as the basis for the analysis. Multiple SAR images of the same locations are processed using ESA Sentinel Application Platform (SNAP) software and merged to remove temporal noise-like artifacts caused by factors such as ships and waves. Next, the iDPolRAD algorithm is employed to detect potential aquaculture sites, which initially include noise from coastal zones and unwanted human and natural structures that pass through the filter. The aquaculture sites are classified using a ResNet18 model with 93% of the sites correctly classified. This implies that it is feasible to monitor marine areas using satellite radar data to track aquaculture areas. However, generalization power across regions is poor likely due to the diversity of types of structures used and species cultivated. Further studies are needed to investigate factors influencing the detectability of different aquaculture sites such as cage geometry or SAR image resolution in order to enhance the accuracy and comprehensiveness of the mapping process. This study highlights the potential of SAR data, coupled with image processing and classification techniques, as a viable means to map large marine areas dedicated to aquaculture.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

CartoCell, a high-content pipeline for 3D image analysis, unveils cell morphology patterns in epithelia

  • Andrés-San Román, Jesús A.
  • Gordillo-Vázquez, Carmen
  • Franco-Barranco, Daniel
  • Morat, Laura
  • Fernández-Espartero, Cecilia H.
  • Baonza, Gabriel
  • Tagua, Antonio
  • Vicente-Munuera, Pablo
  • Palacios, Ana M.
  • Gavilán, María P.
  • Martín-Belmonte, Fernando
  • Annese, Valentina
  • Gómez-Gálvez, Pedro
  • Arganda-Carreras, Ignacio
  • Escudero, Luis M.
Decades of research have not yet fully explained the mechanisms of epithelial self-organization and 3D packing. Single-cell analysis of large 3D epithelial libraries is crucial for understanding the assembly and function of whole tissues. Combining 3D epithelial imaging with advanced deep-learning segmentation methods is essential for enabling this high-content analysis. We introduce CartoCell, a deep-learning-based pipeline that uses small datasets to generate accurate labels for hundreds of whole 3D epithelial cysts. Our method detects the realistic morphology of epithelial cells and their contacts in the 3D structure of the tissue. CartoCell enables the quantification of geometric and packing features at the cellular level. Our single-cell cartography approach then maps the distribution of these features on 2D plots and 3D surface maps, revealing cell morphology patterns in epithelial cysts. Additionally, we show that CartoCell can be adapted to other types of epithelial tissues., This work is supported by the project PID2019-103900GB-I00 funded by MCIN/AEI /10.13039/501100011033 and Programa Operativo FEDER Andalucía 2014–2020 (US-1380953) to L.M.E. Work by L.M.E. and J.A.A.-S.R. has been funded by the Junta de Andalucía (Consejerı´a de economı´a, conocimiento, empresas y Universidad) grant PY18-631 co-funded by FEDER funds. A.T. has been funded by a ‘‘Contrato predoctoral PIF’’ from Universidad de Sevilla. C.G.-V. has been funded by a ‘‘Contrato predoctoral para la formacio´ n de doctores’’ BES-2017-082306. G.B. was supported by a Comunidad de Madrid contract (CAM) and by an FPI grant from MINECO (BES-2022-077789). F.M.-B. was supported by MICINN (PID2020-120367GB-I00) and Fundacio´ n Ramo´ n Areces (CIVP18A3904). P.G.-G. has been funded by Margarita Salas Fellowship – NextGenerationEU. C.H.F.-E. has been funded by Marı´a Zambrano Fellowship – NextGenerationEU. I.A.-C. would like to acknowledge that his work has been partially supported by the University of the Basque Country UPV/EHU grant GIU19/027 and by grant PID2021-126701OB-I00, funded by MCIN/AEI/10.13039/501100011033 and by ‘‘ERDF A way of making Europe." L.M.E. also wants to thank PIE-202120E047 – Conexiones-Life network for networking and input., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

White Blood Cell Classification: Convolutional Neural Network (CNN) and Vision Transformer (ViT) under Medical Microscope

  • Ali, Mohamad Abou
  • Dornaika, Fadi
  • Arganda-Carreras, Ignacio
Deep learning (DL) has made significant advances in computer vision with the advent of vision transformers (ViTs). Unlike convolutional neural networks (CNNs), ViTs use self-attention to extract both local and global features from image data, and then apply residual connections to feed these features directly into a fully networked multilayer perceptron head. In hospitals, hematologists prepare peripheral blood smears (PBSs) and read them under a medical microscope to detect abnormalities in blood counts such as leukemia. However, this task is time-consuming and prone to human error. This study investigated the transfer learning process of the Google ViT and ImageNet CNNs to automate the reading of PBSs. The study used two online PBS datasets, PBC and BCCD, and transferred them into balanced datasets to investigate the influence of data amount and noise immunity on both neural networks. The PBC results showed that the Google ViT is an excellent DL neural solution for data scarcity. The BCCD results showed that the Google ViT is superior to ImageNet CNNs in dealing with unclean, noisy image data because it is able to extract both global and local features and use residual connections, despite the additional time and computational overhead., This work is supported by grant PID2021-126701OB-I00 funded by MCIN/AEI/10.13039/501100011033 and by “ERDF A way of making Europe”, and by grant GIU19/027 funded by the University of the Basque Country UPV/EHU., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Blood Cell Revolution: Unveiling 11 Distinct Types with ‘Naturalize’ Augmentation

  • Ali, Mohamad Abou
  • Dornaika, Fadi
  • Arganda-Carreras, Ignacio
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/)., Artificial intelligence (AI) has emerged as a cutting-edge tool, simultaneously accelerating, securing, and enhancing the diagnosis and treatment of patients. An exemplification of this capability is evident in the analysis of peripheral blood smears (PBS). In university medical centers, hematologists routinely examine hundreds of PBS slides daily to validate or correct outcomes produced by advanced hematology analyzers assessing samples from potentially problematic patients. This process may logically lead to erroneous PBC readings, posing risks to patient health. AI functions as a transformative tool, significantly improving the accuracy and precision of readings and diagnoses. This study reshapes the parameters of blood cell classification, harnessing the capabilities of AI and broadening the scope from 5 to 11 specific blood cell categories with the challenging 11-class PBC dataset. This transformation facilitates a more profound exploration of blood cell diversity, surpassing prior constraints in medical image analysis. Our approach combines state-of-the-art deep learning techniques, including pre-trained ConvNets, ViTb16 models, and custom CNN architectures. We employ transfer learning, fine-tuning, and ensemble strategies, such as CBAM and Averaging ensembles, to achieve unprecedented accuracy and interpretability. Our fully fine-tuned EfficientNetV2 B0 model sets a new standard, with a macro-average precision, recall, and F1-score of 91%, 90%, and 90%, respectively, and an average accuracy of 93%. This breakthrough underscores the transformative potential of 11-class blood cell classification for more precise medical diagnoses. Moreover, our groundbreaking “Naturalize” augmentation technique produces remarkable results. The 2K-PBC dataset generated with “Naturalize” boasts a macro-average precision, recall, and F1-score of 97%, along with an average accuracy of 96% when leveraging the fully fine-tuned EfficientNetV2 B0 model. This innovation not only elevates classification performance but also addresses data scarcity and bias in medical deep learning. Our research marks a paradigm shift in blood cell classification, enabling more nuanced and insightful medical analyses. The “Naturalize” technique’s impact extends beyond blood cell classification, emphasizing the vital role of diverse and comprehensive datasets in advancing healthcare applications through deep learning., This work is supported by grant PID2021-126701OB-I00 funded by MCIN/AEI/10.13039/501100011033 and by “ERDF A way of making Europe”, and by grant GIU19/027 funded by the University of the Basque Country UPV/EHU., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

3D Domain Adaptive Instance Segmentation via Cyclic Segmentation GANs

  • Lauenburg, Leander
  • Lin, Zudi
  • Zhang, Ruihan
  • Santos, Márcia dos
  • Huang, Siyu
  • Arganda-Carreras, Ignacio
  • Boyden, Edward S.
  • Pfister, Hanspeter
  • Wei, Donglai
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/, 3D instance segmentation for unlabeled imaging modalities is a challenging but essential task as collecting expert annotation can be expensive and time-consuming. Existing works segment a new modality by either deploying pre-trained models optimized on diverse training data or sequentially conducting image translation and segmentation with two relatively independent networks. In this work, we propose a novel Cyclic Segmentation Generative Adversarial Network ( CySGAN ) that conducts image translation and instance segmentation simultaneously using a unified network with weight sharing. Since the image translation layer can be removed at inference time, our proposed model does not introduce additional computational cost upon a standard segmentation model. For optimizing CySGAN, besides the CycleGAN losses for image translation and supervised losses for the annotated source domain, we also utilize self-supervised and segmentation-based adversarial objectives to enhance the model performance by leveraging unlabeled target domain images. We benchmark our approach on the task of 3D neuronal nuclei segmentation with annotated electron microscopy (EM) images and unlabeled expansion microscopy (ExM) data. The proposed CySGAN outperforms pre-trained generalist models, feature-level domain adaptation models, and the baselines that conduct image translation and segmentation sequentially. Our implementation and the newly collected, densely annotated ExM zebrafish brain nuclei dataset, named NucExM , are publicly available at https://connectomics-bazaar.github.io/proj/CySGAN/index.html ., The work of Leander Lauenburg was supported by a fellowship through IFI Program of the German Academic Exchange Service (DAAD). The work of Ignacio Arganda-Carreras was supported in part by the University of the Basque Country UPV/EHU under Grant GIU19/027 and in part by Ministerio de Ciencia, Innovación y Universidades, AEI under Grant PID2021-126701OB-I00. The work of Edward S. Boyden was supported by NIH under Grant 1R01EB024261, in part by Lisa Yang, in part by John Doerr, in part by NIH under Grant 1R01MH123403, and in part by NIH under Grant 1R01MH123977, and Schmidt Futures. This work was supported in part by N S F under Grants IIS-1835231 and IIS-2124179, and in part by NIH under Grant 5U54CA225088-03., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Current Progress and Challenges in Large-Scale 3D Mitochondria Instance Segmentation

  • Franco-Barranco, Daniel
  • Lin, Zudi
  • Jang, Won-Dong
  • Wang, Xueying
  • Shen, Qijia
  • Yin, Wenjie
  • Fan, Yutian
  • Li, Mingxing
  • Chen, Chang
  • Xiong, Zhiwei
  • Xin, Rui
  • Liu, Hao
  • Chen, Huai
  • Li, Zhili
  • Zhao, Jie
  • Chen, Xuejin
  • Pape, Constantin
  • Conrad, Ryan
  • Nightingale, Luke
  • Folter, Joost de
  • Jones, Martin L.
  • Liu, Yanling
  • Ziaei, Dorsa
  • Huschauer, Stephan
  • Arganda-Carreras, Ignacio
  • Pfister, Hanspeter
  • Wei, Donglai
© 2023 The Authors. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/, In this paper, we present the results of the MitoEM challenge on mitochondria 3D instance segmentation from electron microscopy images, organized in conjunction with the IEEE-ISBI 2021 conference. Our benchmark dataset consists of two large-scale 3D volumes, one from human and one from rat cortex tissue, which are 1,986 times larger than previously used datasets. At the time of paper submission, 257 participants had registered for the challenge, 14 teams had submitted their results, and six teams participated in the challenge workshop. Here, we present eight top-performing approaches from the challenge participants, along with our own baseline strategies. Posterior to the challenge, annotation errors in the ground truth were corrected without altering the final ranking. Additionally, we present a retrospective evaluation of the scoring system which revealed that: 1) challenge metric was permissive with the false positive predictions; and 2) size-based grouping of instances did not correctly categorize mitochondria of interest. Thus, we propose a new scoring system that better reflects the correctness of the segmentation results. Although several of the top methods are compared favorably to our own baselines, substantial errors remain unsolved for mitochondria with challenging morphologies. Thus, the challenge remains open for submission and automatic evaluation, with all volumes available for download., This work was supported in part by NSF under Award IIS-1835231; in part by NIH under Award 5U54CA225088-03; in part by the University of the Basque Country (UPV/EHU) under Grant GIU19/027; in part by Ministerio de Ciencia, Innovación y Universidades, MCIN/AEI/10.13039/501100011033, under Grant PID2021-126701OB-I00; in part by The Francis Crick Institute which receives its core funding from Cancer Research U.K. under Grant FC001999; in part by the U.K. Medical Research Council under Grant FC001999; in part by the Wellcome Trust under Grant FC001999; and in part by the Frederick National Laboratory for Cancer Research under NIH Contract 75N91019D00024., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Modeling Wound Healing Using Vector Quantized Variational Autoencoders and Transformers

  • Backová, Lenka
  • Bengoetxea, Guillermo
  • Rogalla, Svana
  • Franco-Barranco, Daniel
  • Solon, Jérôme
  • Arganda-Carreras, Ignacio
IEEE 20th International Symposium on Biomedical Imaging (ISBI), 18-21 April 2023, Cartagena, Colombia., Wound healing is a fundamental mechanism for living animals. Understanding the process is crucial for numerous medical applications ranging from scarless healing to faster tissue regeneration and safer post-surgery recovery. In this work, we collect a dataset of time-lapse sequences of Drosophila embryos recovering from a laser-incised wound. We model the wound healing process as a video prediction task for which we utilize a two-stage approach with a vector quantized variational autoencoder and an autoregressive transformer. We show our trained model is able to generate realistic videos conditioned on the initial frames of the healing. We evaluate the model predictions using distortion measures and perceptual quality metrics based on segmented wound masks. Our results show that the predictions keep pixel-level error low while behaving in a realistic manner, thus suggesting the neural network is able to model the wound-closing process., This work is supported in part by grants PID2019-109117GB-100 and PID2021-126701OBI00 funded by MCIN/AEI/10.13039/501100011033 and by “ERDF A way of making Europe”, by the foundation Biofisika Bizkaia and by grant GIU19/027 funded by the UPV/EHU. Author SR is supported by Human Frontier Science Program Organization (fellowship LT0007/2022-L) and author GB is supported by a fellowship from Ministerio de Ciencia e Innovacion (fellowship PRE2020-094463)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Optimizing Video Analytics Deployment for In-Flight Cabin Readiness Verification

  • Elordi, Unai
  • Aranjuelo, Nerea
  • Unzueta, Luis
  • Apellaniz, José Luis
  • Arganda-Carreras, Ignacio
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/, This paper proposes an approach to optimize the deployment of on-board video analytics for checking the correct positioning of luggage in aircraft cabins. The system consists of embedded cameras installed on top of the cabin and a heterogeneous embedded processor. Each camera covers multiple regions of interest (i.e., multiple seats or aisle sections) to minimize the number of cameras required. Each image region is processed by a separate image classification algorithm trained with the expected kind of visual appearance considering the effect of perspective and lens distortion. They classify these regions as correct or incorrect for cabin readiness by exploiting the hierarchical structure of classes composed of different configurations of passengers’ and objects’ presence or absence and the objects’ location. Our approach leverages semantic distances between classes to guide prototypical neural networks for multi-tasking between the main classification (i.e., correct or incorrect status) and auxiliary attributes (i.e., scene configurations), learning robust features from different data domains (i.e., various cabins, real or synthetic). The processing pipeline optimizes response delay and power consumption by leveraging embedded processors’ computing capabilities. We carried out experiments in a cabin mockup with a Jetson AGX Xavier, efficiently obtaining better-quality descriptive information from the scene to improve the system’s accuracy compared to alternative state-of-the-art methods., This work was supported in part by the Clean Sky 2 Joint Undertaking through the European Union’s Horizon 2020 Research and Innovation Program under Grant 865162; in part by Smart Cabin System for cabin readiness COVID Amendment (SmaCS) (https://www.smacs.eu/), from the University of the Basque Country (UPV/EHU) under Grant GIU19/027; and in part by the Ministerio de Ciencia, Innovacion y Universidades, Agencia Estatal de Investigación (AEI), under Grant PID2021-126701OB-I00., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Transformer-based fall detection in videos

  • Núñez-Marcos, Adrián
  • Arganda-Carreras, Ignacio
© 2024 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/), Falls pose a major threat for the elderly as they result in severe consequences for their physical and mental health or even death in the worst-case scenario. Nonetheless, the impact of falls can be alleviated with appropriate technological solutions. Fall detection is the task of recognising a fall, i.e. detecting when a person has fallen in a video. Such an algorithm can be implemented in lightweight devices which can then cater to the users’ needs, e.g. alerting emergency services or caregivers. At the core of those systems, a model capable of promptly recognising falls is crucial for reducing the time until help comes. In this paper we propose a fall detection solution based on transformers, i.e. state-of-the-art neural networks for computer vision tasks. Our model takes a video clip and decides if a fall has occurred or not. In a video stream, it would be applied in a sliding-window fashion to trigger an alarm as soon as it detects a fall. We evaluate our fall detection backbone model on the large UP-Fall dataset, as well as on the UR fall dataset, and compare our results with existing literature using the former dataset., This work is supported by grant PID2021-126701OB-I00 funded by MCIN/AEI/10.13039/501100011033 and by “ERDF A way of making Europe”, and by grant GIU19/027 funded by the University of the Basque Country UPV/EHU ., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Naturalize Revolution: Unprecedented AI-Driven Precision in Skin Cancer Classification Using Deep Learning

  • Ali, Mohamad Abou
  • Dornaika, Fadi
  • Arganda-Carreras, Ignacio
  • Ali, Hussein
  • Karaouni, Malak
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/)., Background: In response to the escalating global concerns surrounding skin cancer, this study aims to address the imperative for precise and efficient diagnostic methodologies. Focusing on the intricate task of eight-class skin cancer classification, the research delves into the limitations of conventional diagnostic approaches, often hindered by subjectivity and resource constraints. The transformative potential of Artificial Intelligence (AI) in revolutionizing diagnostic paradigms is underscored, emphasizing significant improvements in accuracy and accessibility. Methods: Utilizing cutting-edge deep learning models on the ISIC2019 dataset, a comprehensive analysis is conducted, employing a diverse array of pre-trained ImageNet architectures and Vision Transformer models. To counteract the inherent class imbalance in skin cancer datasets, a pioneering “Naturalize” augmentation technique is introduced. This technique leads to the creation of two indispensable datasets—the Naturalized 2.4K ISIC2019 and groundbreaking Naturalized 7.2K ISIC2019 datasets—catalyzing advancements in classification accuracy. The “Naturalize” augmentation technique involves the segmentation of skin cancer images using the Segment Anything Model (SAM) and the systematic addition of segmented cancer images to a background image to generate new composite images. Results: The research showcases the pivotal role of AI in mitigating the risks of misdiagnosis and under-diagnosis in skin cancer. The proficiency of AI in analyzing vast datasets and discerning subtle patterns significantly augments the diagnostic prowess of dermatologists. Quantitative measures such as confusion matrices, classification reports, and visual analyses using Score-CAM across diverse dataset variations are meticulously evaluated. The culmination of these endeavors resulted in an unprecedented achievement—100% average accuracy, precision, recall, and F1-score—within the groundbreaking Naturalized 7.2K ISIC2019 dataset. Conclusion: This groundbreaking exploration highlights the transformative capabilities of AI-driven methodologies in reshaping the landscape of skin cancer diagnosis and patient care. The research represents a pivotal stride towards redefining dermatological diagnosis, showcasing the remarkable impact of AI-powered solutions in surmounting the challenges inherent in skin cancer diagnosis. The attainment of 100% across crucial metrics within the Naturalized 7.2K ISIC2019 dataset serves as a testament to the transformative capabilities of AI-driven approaches in reshaping the trajectory of skin cancer diagnosis and patient care. This pioneering work paves the way for a new era in dermatological diagnostics, heralding the dawn of unprecedented precision and efficacy in the identification and classification of skin cancers., This work is partially supported by grant GIU23/022 unded by the University of the Basque Country (UPV/EHU), and grant PID2021-126701OB-I00, funded by the Ministerio de Ciencia, Innovación y Universidades, AEI, MCIN/AEI/10.13039/501100011033, and by “ERDF A way of making Europe” (to I.A-C.), Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

A deep learning method that identifies cellular heterogeneity using nanoscale nuclear features

  • Carnevali, Davide
  • Zhong, Limei
  • González-Almela, Esther
  • Viana, Carlotta
  • Rotkevich, Mikhail
  • Wang, Aiping
  • Franco-Barranco, Daniel
  • González-Marfil, Aitor
  • Neguembor, Maria Victoria
  • Castells-García, Álvaro
  • Arganda-Carreras, Ignacio
  • Cosma, Maria Pia
Cellular phenotypic heterogeneity is an important hallmark of many biological processes and understanding its origins remains a substantial challenge. This heterogeneity often reflects variations in the chromatin structure, influenced by factors such as viral infections and cancer, which dramatically reshape the cellular landscape. To address the challenge of identifying distinct cell states, we developed artificial intelligence of the nucleus (AINU), a deep learning method that can identify specific nuclear signatures at the nanoscale resolution. AINU can distinguish different cell states based on the spatial arrangement of core histone H3, RNA polymerase II or DNA from super-resolution microscopy images. With only a small number of images as the training data, AINU correctly identifies human somatic cells, human-induced pluripotent stem cells, very early stage infected cells transduced with DNA herpes simplex virus type 1 and even cancer cells after appropriate retraining. Finally, using AI interpretability methods, we find that the RNA polymerase II localizations in the nucleoli aid in distinguishing human-induced pluripotent stem cells from their somatic cells. Overall, AINU coupled with super-resolution microscopy of nuclear structures provides a robust tool for the precise detection of cellular heterogeneity, with considerable potential for advancing diagnostics and therapies in regenerative medicine, virology and cancer biology., We acknowledge support from the European Union’s Horizon 2020 Research and Innovation Programme for two projects (no. 964342 and no. 686637 to M.P.C.); Ministerio de Ciencia e Innovación (grant no. PID2020-114080GB-I00/AEI/10.13039/501100011033 and grant no. BFU2017-86760-P/AEI/FEDER, UE to M.P.C.) and an AGAUR grant from the Departament de Recerca i Universitats de la Generalitat de Catalunya (2021-SGR2021-01300 to M.P.C.); the People Program (Marie Curie Actions) FP7/2007–2013 under REA (grant no. 608959 to M.V.N.); Ministerio de Economia y Competitvidad’s Juan de la Cierva-Incorporación 2017 (IJCI-2017-31831 to M.V.N.); Institucio Catalana de Recerca i Estudis Avançats (ICREA) (to M.P.C.); Ministerio Ciencia e Innovación and ‘ERDF A way of making Europe’ (AEI/10.13039/501100011033 grant PID2021-126701OB-I00 to I.A.-C.); the National Natural Science Foundation of China (31971177 and 32270577 to M.P.C. and 32250410296 to A.C.-G.); and Guangzhou Key Projects of Brain Science and Brain-Like Intelligence Technology (20200730009 to M.P.C.). We acknowledge support of the Spanish Ministry of Science and Innovation through the Centro de Excelencia Severo Ochoa (CEX2020-001049-S, MCIN/AEI/10.13039/501100011033), and the Generalitat de Catalunya through the CERCA programme. We are grateful to the CRG Core Technologies Programme for their support and assistance in this work. We acknowledge D. Pei (GIBH, China) for kindly sharing the urine epithelial cell line; Z. Li and G. Pan (GIBH, China) for kindly sharing the urine-epithelial-cell-derived iPSC; M. A. Sanz and L. Carrasco (Centro de Biologia Molecular Severo Ochoa, CBMSO, Spain) for kindly sharing HSV-1 (KOS strain); and B. Huang (UCSF, USA) for kindly sharing Insight3 software., With funding from the Spanish goverment through the "Severo Ochoa Centre of Excellence" accreditation (CEX2020-001049-S), Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Automatic mapping of aquaculture activity in the Atlantic Ocean

  • Lekunberri, Xabier
  • Ballester-Berman, Josep David
  • Arganda-Carreras, Ignacio
  • Fernandes-Salvador, Jose A.
The production of wild fish has remained relatively stable in the last two decades, whereas aquaculture organism production has increased to the point where it has exceeded wild catches. In this context, accurate and up-to-date information about the current usage of marine areas for aquaculture is crucial for the planning of marine activities. However, this data is often limited to national authorities, and discrepancies between planned and real practices can arise in available data. In this study, a novel methodology to automatically map and verify the current activity of aquaculture crops across European regions based on freely available satellite data is proposed. The European Space Agency's (ESA) Sentinel-1 mission provides Synthetic Aperture Radar (SAR) images, which serve as the basis for the analysis. Multiple SAR images of the same locations are processed using ESA Sentinel Application Platform (SNAP) software and merged to remove temporal noise-like artifacts caused by factors such as ships and waves. Next, the iDPolRAD algorithm is employed to detect potential aquaculture sites, which initially include noise from coastal zones and unwanted human and natural structures that pass through the filter. The aquaculture sites are classified using a ResNet18 model with 93% of the sites correctly classified. This implies that it is feasible to monitor marine areas using satellite radar data to track aquaculture areas. However, generalization power across regions is poor likely due to the diversity of types of structures used and species cultivated. Further studies are needed to investigate factors influencing the detectability of different aquaculture sites such as cage geometry or SAR image resolution in order to enhance the accuracy and comprehensiveness of the mapping process. This study highlights the potential of SAR data, coupled with image processing and classification techniques, as a viable means to map large marine areas dedicated to aquaculture., Xabier Lekunberri has benefited from a Ph.D. grant from the IKERTALENT Programme of the Department of Economic Development, Sustainability, and Environment Department of the Basque Government. This work was partly funded by the European Union’s Horizon 2020 research and innovation program under grant agreements No 101136674 (OptiFish), No 869300 (FutureMARES project) and No 862428 (Mission Atlantic project). It has been also supported by BioBoost + project within the Biodiversa + European Biodiversity Partnership program. Funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or Fundación Biodiversidad. Neither the European Union nor the granting authority can be held responsible for them. The authors wish to express their gratitude to the European Space Agency for providing the SNAP software and for processing and distributing the Sentinel-1 data that was downloaded from the Copernicus Open Access Hub and the Alaska Satellite Facility’s data portal. This work is partially supported by grant GIU19/027 funded by the University of the Basque Country (UPV/EHU), and grant PID2021-126701OB-I00 funded by the Ministerio de Ciencia, Innovación y Universidades, AEI, MCIN/AEI/10.13039/501100011033, and by “ERDF A way of making Europe”. This paper is contribution No 1232 from AZTI, Marine Research, Basque Research and Technology Alliance (BRTA)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Enhancing MRI brain tumor classification: A comprehensive approach integrating real-life scenario simulation and augmentation techniques

  • Ali, Mohamad Abou
  • Dornaika, Fadi
  • Arganda-Carreras, Ignacio
  • Chmouri, Rejdi
  • Shayeh, Hussien
Brain cancer poses a significant global health challenge, with mortality rates showing a concerning surge over recent decades. The incidence of brain cancer-related mortality has risen from 140,000 to 250,000, accompanied by a doubling in new diagnoses from 175,000 to 350,000. In response, magnetic resonance imaging (MRI) has emerged as a pivotal diagnostic tool, facilitating early detection and treatment planning. However, the translation of deep learning approaches to brain cancer diagnosis faces a critical obstacle: the scarcity of public clinical datasets reflecting real-world complexities. This study aims to bridge this gap through a comprehensive exploration and augmentation of training data. Initially, a battery of pre-trained deep models undergoes evaluation on a main brain cancer MRI "BT-MRI" dataset, yielding remarkable performance metrics, including 100% accuracy, precision, recall, and F1-Score, substantiated by the Score-CAM methodology. This initial success underscores the potential of deep learning in brain cancer diagnosis. Subsequently, the model's efficacy undergoes further scrutiny using a supplementary brain cancer MRI "BCD-MRI" dataset, affirming its robustness and applicability across diverse datasets. However, the ultimate litmus test lies in confronting the model with synthetic testing datasets crafted to emulate real-world scenarios. The synthetic testing datasets, a BCD-MRI testing sub-dataset enriched with noise, blur, and simulated patient motion, reveal a sobering reality: the model's performance plummets, exposing inherent limitations in generalization. To address this issue, a diverse set of optimization strategies and augmentation techniques, ranging from diverse optimizers to sophisticated data augmentation methods, are exhaustively explored. Despite these efforts, the problem of generalization persists. The breakthrough emerges with the integration of noise and blur as augmentation techniques during the training process. Leveraging Gaussian noise and Gaussian blur kernels, the model undergoes a transformative evolution, exhibiting newfound robustness and resilience. Retesting the refined model against the challenging synthetic datasets reveals a remarkable transformation, with performance metrics witnessing a notable ascent. This achievement underscores the important role of correct selection of data augmentation in fortifying the generalization of deep learning models for brain cancer diagnosis. This study not only advances the frontiers of diagnostic precision in brain cancer but also underscores the paramount importance of methodological rigor and innovation in confronting the complexities of real-world clinical scenarios., This work has been partially supported by the University of the Basque Country UPV/EHU grant GIU23/022, and by the Ministerio de Ciencia, Innovación y Universidades, AEI , under grant PID2021-126701OB-I00., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Towards Self-Conscious AI Using Deep ImageNet Models: Application for Blood Cell Classification

  • Abou Ali, Mohamad
  • Dornaika, Fadi
  • Arganda-Carreras, Ignacio
The exceptional performance of ImageNet competition winners in image classification has led AI researchers to repurpose these models for a whole range of tasks using transfer learning (TL). TL has been hailed for boosting performance, shortening learning time and reducing computational effort. Despite these benefits, issues such as data sparsity and the misrepresentation of classes can diminish these gains, occasionally leading to misleading TL accuracy scores. This research explores the innovative concept of endowing ImageNet models with a self-awareness that enables them to recognize their own accumulated knowledge and experience. Such self-awareness is expected to improve their adaptability in various domains. We conduct a case study using two different datasets, PBC and BCCD, which focus on blood cell classification. The PBC dataset provides high-resolution images with abundant data, while the BCCD dataset is hindered by limited data and inferior image quality. To compensate for these discrepancies, we use data augmentation for BCCD and undersampling for both datasets to achieve balance. Subsequent pre-processing generates datasets of different size and quality, all geared towards blood cell classification. We extend conventional evaluation tools with novel metrics—“accuracy difference” and “loss difference”—to detect overfitting or underfitting and evaluate their utility as potential indicators for learning behavior and promoting the self-confidence of ImageNet models. Our results show that these metrics effectively track learning progress and improve the reliability and overall performance of ImageNet models in new applications. This study highlights the transformative potential of turning ImageNet models into self-aware entities that significantly improve their robustness and efficiency in various AI tasks. This groundbreaking approach opens new perspectives for increasing the effectiveness of transfer learning in real-world AI implementations., This work is partially supported by grant GIU23/022, funded by the University of the Basque Country (UPV/EHU); grant PID2021-126701OB-I00, funded by the Ministerio de Ciencia, Innovación y Universidades, AEI, MCIN/AEI/10.13039/501100011033; and by “ERDF A way of making Europe”., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Otros (Other)

DL4MicEverywhere: deep learning for microscopy made flexible, shareable and reproducible

  • Hidalgo Cenalmor, Iván
  • Pylvänäinen, Joanna W.
  • Ferreira, Mariana G.
  • Russell, Craig T.
  • Saguy, Alon
  • Arganda-Carreras, Ignacio
  • Shechtman, Yoav
  • Jacquemet, Guillaume
  • Henriques, Ricardo
  • Gómez de Mariscal, Estibaliz
I.H.-C., M.G.F., C.T.R., R.H. and E.G.-d.-M. received funding from the European Union through the Horizon Europe program (AI4LIFE project with grant agreement 101057970-AI4LIFE and RT-SuperES project with grant agreement 101099654-RTSuperES to R.H.). I.H.-C., M.G.F., E.G.-d.-M. and R.H. also acknowledge the support of the Gulbenkian Foundation (Fundação Calouste Gulbenkian) and the European Research Council (ERC) under the European Union’s Horizon 2020 Research and Innovation Programme (grant agreement 101001332 to R.H.). Funded by the European Union. Views and opinions expressed are, however, those of the authors only and do not necessarily reflect those of the European Union. Neither the European Union nor the granting authority can be held responsible for them. This work was also supported by a European Molecular Biology Organization (EMBO) installation grant (EMBO-2020-IG-4734 to R.H.), an EMBO postdoctoral fellowship (EMBO ALTF 174-2022 to E.G.-d.-M.), a Chan Zuckerberg Initiative Visual Proteomics Grant (vpi-0000000044 with https://doi.org/10.37921/743590vtudfp to R.H.). R.H. also acknowledges the support of LS4FUTURE Associated Laboratory (LA/P/0087/2020). This work is partially supported by grant GIU19/027 (to I.A.-C.) funded by the University of the Basque Country (UPV/EHU), grant PID2021-126701OB-I00 (to I.A.-C.) funded by the Ministerio de Ciencia, Innovación y Universidades, MICIU/AEI/10.13039/501100011033, and “ERDF A way of making Europe” (to I.A.-C.). This study was also supported by the Academy of Finland (338537 to G.J.), the Sigrid Juselius Foundation (to G.J.), the Cancer Society of Finland (Syöpäjärjestöt; to G.J.), and Solutions for Health strategic funding to Åbo Akademi University (to G.J.). This research was supported by the InFLAMES Flagship Programme of the Academy of Finland (decision number 337531)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Otros (Other)

BiaPy: accessible deep learning on bioimages

  • Franco-Barranco, Daniel
  • Andrés-San Román, Jesús A.
  • Hidalgo Cenalmor, Iván
  • Backová, Lenka
  • González-Marfil, Aitor
  • Caporal, Clément
  • Chessel, Anatole
  • Gómez-Gálvez, Pedro
  • Escudero, Luis M.
  • Wei, Donglai
  • Muñoz-Barrutia, Arrate
  • Arganda-Carreras, Ignacio
Bioimage analysis is a cornerstone of modern life sciences, powering discoveries and insights derived from biological image data. Deep learning has become an invaluable tool for analyzing microscopy datasets, and its application is increasingly widespread in biomedical research1. However, its prerequisite for high-level programming skills has often acted as a barrier, limiting accessibility for researchers without a specific computational background. [...], This work is partially supported by grant GIU23/022 (to I.A.-C.) funded by the University of the Basque Country (UPV/EHU), grants PID2021-126701OB-I00 (to I.A-C.) and PID2023152631OB-I00 (to A.M.-B.), funded by the Ministerio de Ciencia, Innovación y Universidades, AEI, MCIN/AEI/10.13039/501100011033, and by “ERDF A way of making Europe”. P.G.G. has been funded by Margarita Salas Fellowship – NextGenerationEU. J.A.A.-S.R.’s work has been funded by the Junta de Andalucía (Consejería de economía, conocimiento, empresas y Universidad) grant PY18-631 co-funded by FEDER funds. L.M.E. thanks the PIE-202120E047-Conexiones-Life network for its networking and input. I.H-C. and A.M-B. received funding from the European Union through the Horizon Europe program (AI4LIFE project with grant agreement 101057970-AI4LIFE). Funded by the European Union. Views and opinions expressed are, however, those of the authors only and do not necessarily reflect those of the European Union. Neither the European Union nor the granting authority can be held responsible for them. I.H-C. acknowledges the support of the Gulbenkian Foundation (Fundação Calouste Gulbenkian)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Recognizing past shapes: sex differentiation through deep learning on European Upper Palaeolithic hand stencils

  • Fernández Navarro, Verónica
  • González-Marfil, Aitor
  • Arganda-Carreras, Ignacio
  • Garate, Diego
This study explores the application of advanced deep learning techniques in analyzing Upper Palaeolithic hand stencil representations, focusing on sex classification of individuals involved in prehistoric rock art activity. The research highlights the effectiveness of deep learning models, particularly EfficientNetV2-S, which achieved an accuracy rate of 81.03 % for experimental blown hand stencils and 95.08 % in delineated contemporary hand image samples for sex identification, surpassing traditional morphometric methods. The study demonstrates that deep learning can differentiate male and female hand stencils with high precision, suggesting a mixed-sexual participation in creating prehistoric art, with a slight prevalence of male hand representations in the studied caves. The integration of user-friendly platforms, such as Google Colab, facilitates the reproducibility and validation of these findings, promoting methodological transparency. However, the accuracy of deep learning models is contingent on the quality and preservation of the images, presenting challenges when working with deteriorated or incomplete samples. This work highlights the potential of advanced technologies in archaeological research, opening new avenues for investigating the creation of prehistoric graphic expressions and their social implications., The author V. Fernandez-Navarro is beneficiary of a “Concepcion Arenal” 2020–21 pre-doctoral contract at the University of Cantabria for research towards the PhD. The present study has been conducted as part of the research project “Scientific virtual reality for the study and dissemination of the scenarios of artistic creation in Palaeolithic caves (RealCaveART)” (PDC2022-133124-I00), PI: Diego Garate, funded by MCIN/AEI/https://doi.org/10.13039/50110 00110 33 and the European Union Next Generation EU/PRTR. This study is also a part of the research project “From the Mind to the Wall: Tracing Sign Language in Palæolithic Hand Stencils (MIND2WALL)” (ANR-21-CE27-0005), funded by the French Ministry of Culture, PI: Aritz Irurtzun. This work is partially supported by grant GIU23/022 (to I.A-C.) funded by the University of the Basque Country (UPV/EHU) and grant PID2021-126701OB-I00 (to I.A-C.) funded by the Ministerio de Ciencia, Innovación y Universidades, AEI, MCIN/AEI/10.13039/501100011033, and by "ERDF A way of making Europe”., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Enhancing Generalization and Mitigating Overfitting in Deep Learning for Brain Cancer Diagnosis from MRI

  • Ali, Mohamad Abou
  • Charafeddine, Jinan
  • Dornaika, Fadi
  • Arganda-Carreras, Ignacio
Brain cancer represents a significant global health challenge with increasing incidence and mortality rates. Magnetic Resonance Imaging (MRI) plays a pivotal role in early detection and treatment planning. This study adopts a systematic approach across four phases: (1) Optimal Model Selection using the Adam optimizer, emphasizing accuracy metrics, weight computation, early stopping, and ReduceLROnPlateau techniques. (2) Real-world Scenario Simulation through synthetic perturbed datasets created by applying noise, blur (to simulate various magnetic field strengths: 1T, 1.5T, 3T), and patient motion artifacts (mimicking MRI scanning motion effects) to the testing data from the BT-MRI dataset, an online published brain tumor MRI dataset. (3) Optimization involving a range of optimizers (Adam, Adagrad, Nadam, RMSprop, SGD) and online augmentation techniques (AutoMix, CutMix, LGCOAMix, PatchUp). (4) Solution Exploration integrating Gaussian Noise and Blur as augmentation strategies during training to enhance model generalization under diverse conditions. Initial evaluations achieved strong performance, consistently reaching 99.45% accuracy on the BT-MRI dataset. However, testing against synthetic perturbed datasets mimicking real-world conditions revealed challenges in maintaining robust model performance. Despite employing diverse optimization methods and advanced augmentation techniques, this study identifies persistent challenges in ensuring model robustness with synthetic perturbed datasets. Notably, the integration of Gaussian Noise and Blur during training significantly improved model resilience. This research underscores the critical role of methodological rigor and innovative augmentation strategies in advancing deep learning applications for precise brain cancer diagnosis using MRI., This work has been partially supported by the University of the Basque Country UPV/EHU grant GIU23/022, and by the Ministerio de Ciencia, Innovación y Universidades, AEI, under grant PID2021-126701OB-I00., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Deep learning for accurate vision-based catch composition in tropical tuna purse seiners

  • Lekunberri, Xabier
  • Kamal, Ahmad
  • Goienetxea, Izaro
  • Ruiz, Jon
  • Quincoces, Iñaki
  • Valls Miro, Jaime
  • Arganda-Carreras, Ignacio
  • Fernandes-Salvador, José A.
23 pages, 5 figures, Purse seiners play a crucial role in tuna fishing, as approximately 69% of the world’s tropical tuna is caught using this gear. All tuna Regional Fisheries Management Organizations have established minimum standards for the use of electronic monitoring (EM) in fisheries in addition to traditional observers. These EM systems produce a massive amount of video data that human analysts must process. Integrating artificial intelligence (AI) into their workflow can decrease that workload and improve the accuracy of the reports. However, species identification still poses significant challenges for AI, as achieving balanced performance across all species requires appropriate training data. Here, we quantify the difficulty experts face to distinguish bigeye tuna (BET, Thunnus obesus) from yellowfin tuna (YFT, Thunnus albacares) using images captured by EM systems. We found inter-expert agreements of 42.9% ± 35.6% for BET and 57.1% ± 35.6% for YFT. We then present a multi-stage pipeline to estimate the species composition of the catches using a reliable ground-truth dataset based on identifications made by observers on board. Three segmentation approaches are compared: Mask R-CNN, a combination of DINOv2 with SAM2, and an integration of YOLOv9 with SAM2. We found that the latter performs the best, with a validation mean average precision of 0.66 ± 0.03 and a recall of 0.85 ± 0.03. Segmented individuals are tracked using ByteTrack. For classification, we evaluated a standard multiclass classification model and a hierarchical approach. Their performance depended on the fishing trip considered, suggesting that both approaches are promising, but additional testing is needed to determine which strategy generalizes better overall. All our models were cross-validated during training and tested on fishing operations with fully known catch composition. Combining YOLOv9-SAM2 with the hierarchical classification produced the best estimations, with 84.8% of the individuals being segmented and classified with a mean absolute error of 4.5%., X.L. has received a PhD grant from the IKERTALENT Programme of the Department of Economic Development, Sustainability, and Environment of the Basque Government. Open Access funding was provided by the University of the Basque Country (EHU). This work is partially supported by grant GIU23/022 funded by the University of the Basque Country (EHU), and grant PID2021-126701OB-I00, funded by the Ministerio de Ciencia, Innovación y Universidades, AEI, MCIN/AEI/10.13039/501100011033, and by "ERDF A way of making Europe.” This work has been funded by the European Union through the Horizon Europe program projects EVERYFISH (Grant agreement number 101059892) and OptiFish (Grant Agreement number 101136674) as well as BioBoost + project within the Biodiversa + European Biodiversity Partnership program., Peer reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

A deep learning method that identifies cellular heterogeneity using nanoscale nuclear features

  • Carnevali, Davide
  • Zhong, Limei
  • González-Almela, Esther
  • Viana, Carlotta
  • Rotkevich, Mikhail
  • Wang, Aiping
  • Franco-Barranco, Daniel
  • González-Marfil, Aitor
  • Neguembor, Maria Victoria
  • Castells García, Àlvaro, 1991-
  • Arganda-Carreras, Ignacio
  • Cosma, Maria Pia
Cellular phenotypic heterogeneity is an important hallmark of many biological processes and understanding its origins remains a substantial challenge. This heterogeneity often reflects variations in the chromatin structure, influenced by factors such as viral infections and cancer, which dramatically reshape the cellular landscape. To address the challenge of identifying distinct cell states, we developed artificial intelligence of the nucleus (AINU), a deep learning method that can identify specific nuclear signatures at the nanoscale resolution. AINU can distinguish different cell states based on the spatial arrangement of core histone H3, RNA polymerase II or DNA from super-resolution microscopy images. With only a small number of images as the training data, AINU correctly identifies human somatic cells, human-induced pluripotent stem cells, very early stage infected cells transduced with DNA herpes simplex virus type 1 and even cancer cells after appropriate retraining. Finally, using AI interpretability methods, we find that the RNA polymerase II localizations in the nucleoli aid in distinguishing human-induced pluripotent stem cells from their somatic cells. Overall, AINU coupled with super-resolution microscopy of nuclear structures provides a robust tool for the precise detection of cellular heterogeneity, with considerable potential for advancing diagnostics and therapies in regenerative medicine, virology and cancer biology., We acknowledge support from the European Union’s Horizon 2020 Research and Innovation Programme for two projects (no. 964342 and no. 686637 to M.P.C.); Ministerio de Ciencia e Innovación (grant no. PID2020-114080GB-I00/AEI/10.13039/501100011033 and grant no. BFU2017-86760-P/AEI/FEDER, UE to M.P.C.) and an AGAUR grant from the Departament de Recerca i Universitats de la Generalitat de Catalunya (2021-SGR2021-01300 to M.P.C.); the People Program (Marie Curie Actions) FP7/2007–2013 under REA (grant no. 608959 to M.V.N.); Ministerio de Economia y Competitvidad’s Juan de la Cierva-Incorporación 2017 (IJCI-2017-31831 to M.V.N.); Institucio Catalana de Recerca i Estudis Avançats (ICREA) (to M.P.C.); Ministerio Ciencia e Innovación and ‘ERDF A way of making Europe’ (AEI/10.13039/501100011033 grant PID2021-126701OB-I00 to I.A.-C.); the National Natural Science Foundation of China (31971177 and 32270577 to M.P.C. and 32250410296 to A.C.-G.); and Guangzhou Key Projects of Brain Science and Brain-Like Intelligence Technology (20200730009 to M.P.C.). We acknowledge support of the Spanish Ministry of Science and Innovation through the Centro de Excelencia Severo Ochoa (CEX2020-001049-S, MCIN/AEI/10.13039/501100011033), and the Generalitat de Catalunya through the CERCA programme. We are grateful to the CRG Core Technologies Programme for their support and assistance in this work. We acknowledge D. Pei (GIBH, China) for kindly sharing the urine epithelial cell line; Z. Li and G. Pan (GIBH, China) for kindly sharing the urine-epithelial-cell-derived iPSC; M. A. Sanz and L. Carrasco (Centro de Biologia Molecular Severo Ochoa, CBMSO, Spain) for kindly sharing HSV-1 (KOS strain); and B. Huang (UCSF, USA) for kindly sharing Insight3 software.