Pasar al contenido principal

APRENDIZAJE PROFUNDO EFICIENTE PARA SECUENCIAS DE VIDEO Y NUBES DE PUNTOS

PID2020-117142GB-I00

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal de I+D+i Orientado a los Retos de la Sociedad
Convocatoria Proyectos de I+D+i (Generación de Conocimiento y Retos Investigación)
Año convocatoria 2020
Plan Estatal 2017-2020 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario UNIVERSITAT POLITECNICA DE CATALUNYA

Resultados relacionados

Resultados totales (Incluyendo duplicados): 20
Encontrada(s) 1 página(s)

Repositori Obert UdL
Publicación Contribución de congreso (ConferenceOutput)

Amodal segmentation for on-tree apple fruit size es timation with RGB-D images

  • Gené Mola, Jordi
  • Gregorio López, Eduard
  • Ferrer Ferrer, Mar
  • Blok, Pieter
  • Hemming, Jochen
  • Morros Rubió, Josep Ramon
  • Rosell Polo, Joan Ramon
  • Vilaplana Besler, Verónica
  • Ruiz Hidalgo, Javier
The detection and sizing of fruits with computer vision methods is of interest because it provides relevant information to improve the management of orchard farming. However, the presence of partially occluded fruits limits the performance of existing methods, making reliable fruit sizing a challenging task. While previous fruit segmentation works limit segmentation to the visible region of fruits (known as modal segmentation), in this work we propose an amodal segmentation algorithm to predict the complete shape, which includes its visible and occluded regions. CONCLUSIONS The main advantages of the present methodology are its robustness for measuring partially occluded fruits and the capability to determine the visibility percentage. Future works should evaluate the performance of the method with commercial RGB-D sensors, which would facilitate data collection., This work was partly funded by the Departament de Recerca i Universitats de la Generalitat de Catalunya (grant 2021 LLAV 00088), and the Spanish Minis try of Science, Innovation and Universities (grants PID2021-126648OB-I00 [PAgPROTECT] and PID2020-117142GB-I00 [DeeLight] by MCIN/ AEI/10.13039/501100011033 and by “ERDF, a way of making Europe”, by the European Union).
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Real-time lane classification and accident detection for safer micromobility

  • Morros Rubió, Josep Ramon|||0000-0002-1395-487X
  • Broquetas Ibars, Antoni|||0000-0001-9801-9145
  • Mateo i Remacha, Alex
  • Puig Rabat, Jordi
  • Davins Jovells, Manel
The lack of knowledge of the micromobility regulations by e-scooter users is an important factor behind some of the accidents involving these vehicles. We present two modules that can increase the safety of the users and pedestrians: First, a computer vision model that analyses the video feed captured with a smartphone attached to the e-scooter, and predicts in real-time the type of lane in which the user is riding. This knowledge is used by an application which combines this information with GSNN location information and a database of mobility regulations, and informs the user when he/she is not complying with these regulations. Second, an accident detection system, using the smartphone accelerometer, that detects if there is a fall during the riding, so that the app can contact the authorities to determine the appropriate response. The experimental results show excellent results for both modules., We sincerely acknowledge the financial support that funding bodies have provided for this work. We thank the Spanish Ministry of Science, Innovation and Universities (PID2020-117142GB-I00 [DeeLight project] by MCIN/AEI/10.13039/501100011033). Additionally, we extend our gratitude to the EIT Urban Mobility, an initiative of the European Institute of Innovation and Technology (EIT), a body of the European Union, for their support under the RideSafeUM project, Peer Reviewed, Objectius de Desenvolupament Sostenible::11 - Ciutats i Comunitats Sostenibles, Objectius de Desenvolupament Sostenible::9 - Indústria, Innovació i Infraestructura
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Video-based fruit detection and tracking for apple counting and mapping

  • Gené Mola, Jordi
  • Felip Pomés, Marc
  • Net Barnés, Francesc
  • Morros Rubió, Josep Ramon|||0000-0002-1395-487X
  • Miranda, Juan C.
  • Arnó Satorra, Jaume
  • Asín Jones, Luis
  • Lordan Sanahuja, Jaume
  • Ruiz Hidalgo, Javier|||0000-0001-6774-685X
  • Gregorio López, Eduard
Automatic fruit counting systems have garnered interest from farmers and agronomists to monitor fruit production, predict yields in advance, and identify production variability across orchards. However, accurately counting fruits poses challenges, particularly due to occlusions. This study proposes a multi-view sensing approach using continuous motion videos captured by a camera moved along the row of trees, followed by fruit detection in all video frames and application of Multi-Object Tracking (MOT) algorithms to prevent double-counting. Three tracking methods, namely SORT, DeepSORT, and ByteTrack, are compared for fruit counting using the YOLOv5x object detector. The methodology is applied to map fruit production in an experimental apple orchard at two different dates: four weeks and one week before harvest. The results demonstrate that ByteTrack (MOTA=0.682; IDF1=0.837; HOTA=0.689) outperforms SORT and DeepSORT, indicating its superior tracking performance. Computational efficiency analysis reveals similar processing times between SORT and ByteTrack (about 15 ms), while DeepSORT requires significantly more processing time per image (128 ms). Fruit counting evaluation shows reasonably accurate yield predictions on both dates, with reduced errors and improved performance closer to the harvest date (MAPE=7.47 %; R2=0.70). The system proves effective in estimating orchard fruit production using computer vision technology, offering valuable insights for yield forecasting. These findings contribute to optimizing fruit production and supporting precision agriculture practices. The code and the dataset have been made publicly available and a video visualization of results is accessible at http://www.grap.udl.cat/en/publications/video_fruit_counting., This work was partly funded by the Departament de Recerca i Universitats de la Generalitat de Catalunya (grant 2021 LLAV 00088), the Spanish Ministry of Science, Innovation and Universities (grants RTI2018-094222-B-I00 [PAgFRUIT project], PID2021-126648OB-I00 [PAgPROTECT project] and PID2020-117142GB-I00 [DeeLight project] by MCIN/AEI/10.13039/501100011033 and by “ERDF, a way of making Europe”, by the European Union). The work of Jordi Gené Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU. The Secretariat of Universities and Research of the Department of Business and Knowledge of the Generalitat de Catalunya and European Social Fund (ESF) are also thanked for financing Juan Carlos Miranda’s predoctoral fellowship (2020 FI_B 00586)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Enhancing Ki-67 cell segmentation with Dual U-Net models: a step towards uncertainty-informed active learning

  • Anglada Rotger, David|||0009-0009-9792-5889
  • Sala Prat, Júlia
  • Marqués Acosta, Fernando|||0000-0001-8311-1168
  • Salembier Clairon, Philippe Jean|||0000-0001-8884-9604
  • Pardàs Feliu, Montse|||0000-0002-5861-6356
The diagnosis and prognosis of breast cancer relies on histopathology image analysis where markers such as Ki-67 are increasingly important. The diagnosis using this marker is based on quantification of proliferation, which implies counting of Ki-67 positive and negative tumoral cells, excluding stromal cells. A common problem for automatic quantification of these images derives from overlapping and clustering of cells. We propose in this paper an automatic segmentation and classification system that overcomes this problem using two Convolutional Neural Networks (Dual U-Net), whose results are combined with a watershed algorithm. Taking into account that a major issue for the development of reliable neural networks is the availability of labeled databases, we also introduce an approach for epistemic uncertainty estimation that can be used for active learning in instance segmentation applications. We use Monte Carlo Dropout within our networks to quantify the model's confidence across its predictions, offering insights into areas of high uncertainty. Our results show how the postprocessed uncertainty maps can be used to refine ground truth annotations and to generate new labeled data with reduced annotation effort. To initialize the labeling and further reduce this effort, we propose a tool for groundtruth generation which is based on candidate generation with maxtree. Candidates are filtered based on extracted features which can be adjusted for the specific image typology, thereby facilitating precise model training and evaluation., This reseach has been funded by the project PID2020-116907RB-100 [AIMING] and PID2020-117142GB-I00 [DeeLight] by MCIN/ AEI /10.13039/ 501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

SkinningNet: two-stream graph convolutional neural network for skinning prediction of synthetic characters

  • Mosella Montoro, Albert
  • Ruiz Hidalgo, Javier|||0000-0001-6774-685X
This work presents SkinningNet, an end-to-end Two-Stream Graph Neural Network architecture that computes skinning weights from an input mesh and its associated skeleton, without making any assumptions on shape class and structure of the provided mesh. Whereas previous methods pre-compute handcrafted features that relate the mesh and the skeleton or assume a fixed topology of the skeleton, the proposed method extracts this information in an end-to-end learnable fashion by jointly learning the best relationship between mesh vertices and skeleton joints. The proposed method exploits the benefits of the novel Multi-Aggregator Graph Convolution that combines the results of different aggregators during the summarizing step of the Message-Passing scheme, helping the operation to generalize for unseen topologies. Experimental results demonstrate the effectiveness of the contributions of our novel architecture, with SkinningNet outperforming current state-of-the-art alternatives., This work has been partially supported by the project PID2020-117142GB-I00, funded by MCIN/AEI /10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Channel redundancy and overlap in convolutional neural networks with channel-wise NNK graphs

  • Bonet Solé, David
  • Ortega, Antonio
  • Ruiz Hidalgo, Javier|||0000-0001-6774-685X
  • Shekkizhar, Sarath
Feature spaces in the deep layers of convolutional neural networks (CNNs) are often very high-dimensional and difficult to inter-pret. However, convolutional layers consist of multiple channels that are activated by different types of inputs, which suggests that more insights may be gained by studying the channels and how they relate to each other. In this paper, we first analyze theoretically channel-wise non-negative kernel (CW-NNK) regression graphs, which allow us to quantify the overlap between channels and, indirectly, the intrinsic dimension of the data representation manifold. We find that redundancy between channels is significant and varies with the layer depth and the level of regularization during training. Additionally, we observe that there is a correlation between channel overlap in the last convolutional layer and generalization performance. Our experimental results demonstrate that these techniques can lead to a better understanding of deep representations., This work was partially supported by the Spanish Research Agency (AEI) under project PID2020-117142GB-I00 of the call MCIN/ AEI /10.13039/501100011033, and by a DARPA grant (FA8750-19-2-1005) in Learning with Less Labels (LwLL) program and by NSF (CCF-2009032)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Pixinwav: Residual steganography for hiding pixels in audio

  • Geleta Geleta, Margarita
  • Puntí Álvarez, Cristina
  • McGuinness, Kevin
  • Pons Puig, Jordi
  • Canton Ferrer, Cristian
  • Giró Nieto, Xavier|||0000-0002-9935-5332
Steganography comprises the mechanics of hiding data in a host media that may be publicly available. While previous works focused on unimodal setups (e.g., hiding images in images, or hiding audio in audio), PixInWav targets the multimodal case of hiding images in audio. To this end, we propose a novel residual architecture operating on top of short-time discrete cosine transform (STDCT) audio spectrograms. Among our results, we find that the residual steganography setup we propose allows an encoding of the hidden image that is independent from the host audio without compromising quality. Accordingly, while previous works require both host and hidden signals to hide a signal, PixInWav can encode images offline—which can be later hidden, in a residual fashion, into any audio signal., Work partially supported by the European Union through the Erasmus+ student mobility program, Science Foundation Ireland (SFI) under grant numbers SFI/15/SIRG/3283 and SFI/12/RC/2289 P2, and the Spanish Research Agency (AEI) under project PID2020117142GB-I00 of the call MCIN/ AEI /10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

H3D-Net: Few-shot high-fidelity 3D head reconstruction

  • Ramon Maldonado, Eduard
  • Triginer Garcés, Gil
  • Escurt i Gelabert, Janna
  • Pumarola Peris, Albert|||0000-0003-4185-6991
  • García Giráldez, Jaime
  • Giró Nieto, Xavier|||0000-0002-9935-5332
  • Moreno-Noguer, Francesc
Recent learning approaches that implicitly represent surface geometry using coordinate-based neural representations have shown impressive results in the problem of multi-view 3D reconstruction. The effectiveness of these techniques is, however, subject to the availability of a large number (several tens) of input views of the scene, and computationally demanding optimizations. In this paper, we tackle these limitations for the specific problem of few-shot full 3D head reconstruction, by endowing coordinate-based representations with a probabilistic shape prior that enables faster convergence and better generalization when using few input images (down to three). First, we learn a shape model of 3D heads from thousands of incomplete raw scans using implicit representations. At test time, we jointly overfit two coordinate-based neural networks to the scene, one modeling the geometry and another estimating the surface radiance, using implicit differentiable rendering. We devise a two-stage optimization strategy in which the learned prior is used to initialize and constrain the geometry during an initial optimization phase. Then, the prior is unfrozen and fine-tuned to the scene. By doing this, we achieve high-fidelity head reconstructions, including hair and shoulders, and with a high level of detail that consistently outperforms both state-of-the-art 3D Morphable Models methods in the few-shot scenario, and non-parametric methods when large sets of views are available., This work has been partially funded by the Spanish government with the projects MoHuCo PID2020-120049RBI00, DeeLight PID2020-117142GB-I00 and Maria de Maeztu Seal of Excellence MDM-2016-0656, and by the Government of Catalonia under 2017 DI 028., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Fast unsupervised tensor restoration via low-rank deconvolution

  • Reixach i Pérez, David
  • Morros Rubió, Josep Ramon|||0000-0002-1395-487X
Low-rank Deconvolution (LRD) has appeared as a new multi-dimensional representation model that enjoys important efficiency and flexibility properties. In this work we ask ourselves if this analytical model can compete against Deep Learning (DL) frameworks like Deep Image Prior (DIP) or Blind-Spot Networks (BSN) and other classical methods in the task of signal restoration. More specifically, we propose to extend LRD with differential regularization. This approach allows us to easily incorporate Total Variation (TV) and integral priors to the formulation leading to considerable performance tested on signal restoration tasks such image denoising and video enhancement, and at the same time benefiting from its small computational cost., This work has been partially supported by the Spanish Ministry of Science and Innovation under the project DeeLight PID2020-117142GB-I00., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Sign language video retrieval with free-form textual queries

  • Cardoso Duarte, Amanda
  • Albanie, Samuel
  • Giró Nieto, Xavier|||0000-0002-9935-5332
  • Varol, Gül
Systems that can efficiently search collections of sign language videos have been highlighted as a useful application of sign language technology. However, the problem of searching videos beyond individual keywords has received limited attention in the literature. To address this gap, in this work we introduce the task of sign language retrieval with textual queries: given a written query (e.g. a sentence) and a large collection of sign language videos, the objective is to find the signing video that best matches the written query. We propose to tackle this task by learning cross-modal embeddings on the recently introduced large-scale How2Sign dataset of American Sign Language (ASL). We identify that a key bottleneck in the performance of the system is the quality of the sign video embedding which suffers from a scarcity of labelled training data. We, therefore, propose SPOT-ALIGN, a framework for interleaving iterative rounds of sign spotting and feature alignment to expand the scope and scale of available training data. We validate the effectiveness of SPOT-ALIGN for learning a robust sign video embedding through improvements in both sign recognition and the proposed video retrieval task., This work was supported by the project PID2020-117142GB-I00, funded by MCIN/ AEI /10.13039/501100011033, ANR project CorVis ANR-21-CE23-0003- 01, and gifts from Google and Adobe. AD received support from la Caixa Foundation (ID 100010434), fellowship code LCF/BQ/IN18/11660029., Peer Reviewed, Objectius de Desenvolupament Sostenible::10 - Reducció de les Desigualtats, Objectius de Desenvolupament Sostenible::10 - Reducció de les Desigualtats::10.2 - Per a 2030, potenciar i promoure la inclusió social, econòmica i política de totes les persones, independentment de l’edat, sexe, discapacitat, raça, ètnia, origen, religió, situació econòmica o altra condició
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Amodal segmentation for on-tree apple fruit size es timation with RGB-D images

  • Gené Mola, Jordi
  • Gregorio López, Eduard
  • Ferrer Ferrer, Mar
  • Blok, Pieter
  • Hemming, Jochen
  • Morros Rubió, Josep Ramon
  • Rosell Polo, Joan Ramon
  • Vilaplana Besler, Verónica
  • Ruiz Hidalgo, Javier
The detection and sizing of fruits with computer vision methods is of interest because it provides relevant information to improve the management of orchard farming. However, the presence of partially occluded fruits limits the performance of existing methods, making reliable fruit sizing a challenging task. While previous fruit segmentation works limit segmentation to the visible region of fruits (known as modal segmentation), in this work we propose an amodal segmentation algorithm to predict the complete shape, which includes its visible and occluded regions. CONCLUSIONS The main advantages of the present methodology are its robustness for measuring partially occluded fruits and the capability to determine the visibility percentage. Future works should evaluate the performance of the method with commercial RGB-D sensors, which would facilitate data collection., This work was partly funded by the Departament de Recerca i Universitats de la Generalitat de Catalunya (grant 2021 LLAV 00088), and the Spanish Minis try of Science, Innovation and Universities (grants PID2021-126648OB-I00 [PAgPROTECT] and PID2020-117142GB-I00 [DeeLight] by MCIN/ AEI/10.13039/501100011033 and by “ERDF, a way of making Europe”, by the European Union).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Video-based fruit detection and tracking for apple counting and mapping

  • Gené Mola, Jordi
  • Felip Pomés, Marc
  • Net Barnés, Francesc
  • Morros Rubió, Josep Ramon
  • Miranda, Juan C.
  • Arnó Satorra, Jaume
  • Asín Jones, Luis
  • Lordan Sanahuja, Jaume
  • Ruiz Hidalgo, Javier
  • Gregorio López, Eduard
Automatic fruit counting systems have garnered interest from farmers and agronomists to monitor fruit production, predict yields in advance, and identify production variability across orchards. However, accurately counting fruits poses challenges, particularly due to occlusions. This study proposes a multi-view sensing approach using continuous motion videos captured by a camera moved along the row of trees, followed by fruit detection in all video frames and application of Multi-Object Tracking (MOT) algorithms to prevent double-counting. Three tracking methods, namely SORT, DeepSORT, and ByteTrack, are compared for fruit counting using the YOLOv5x object detector. The methodology is applied to map fruit production in an experimental apple orchard at two different dates: four weeks and one week before harvest. The results demonstrate that ByteTrack (MOTA=0.682; IDF1=0.837; HOTA=0.689) outperforms SORT and DeepSORT, indicating its superior tracking performance. Computational efficiency analysis reveals similar processing times between SORT and ByteTrack (about 15 ms), while DeepSORT requires significantly more processing time per image (128 ms). Fruit counting evaluation shows reasonably accurate yield predictions on both dates, with reduced errors and improved performance closer to the harvest date (MAPE=7.47 %; R2=0.70). The system proves effective in estimating orchard fruit production using computer vision technology, offering valuable insights for yield forecasting. These findings contribute to optimizing fruit production and supporting precision agriculture practices. The code and the dataset have been made publicly available and a video visualization of results is accessible at http://www.grap.udl.cat/en/publications/video_fruit_counting., This work was partly funded by the Departament de Recerca i Universitats de la Generalitat de Catalunya (grant 2021 LLAV 00088), the Spanish Ministry of Science, Innovation and Universities (grants RTI2018-094222-B-I00 [PAgFRUIT project], PID2021-126648OB-I00 [PAgPROTECT project] and PID2020-117142GB-I00 [DeeLight project] by MCIN/AEI/10.13039/501100011033 and by “ERDF, a way of making Europe”, by the European Union). The work of Jordi Gené Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU. The Secretariat of Universities and Research of the Department of Business and Knowledge of the Generalitat de Catalunya and European Social Fund (ESF) are also thanked for financing Juan Carlos Miranda’s predoctoral fellowship (2020 FI_B 00586)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Fast unsupervised tensor restoration via low-rank deconvolution

  • Reixach i Pérez, David
  • Morros Rubió, Josep Ramon
Low-rank Deconvolution (LRD) has appeared as a new multi-dimensional representation model that enjoys important efficiency and flexibility properties. In this work we ask ourselves if this analytical model can compete against Deep Learning (DL) frameworks like Deep Image Prior (DIP) or Blind-Spot Networks (BSN) and other classical methods in the task of signal restoration. More specifically, we propose to extend LRD with differential regularization. This approach allows us to easily incorporate Total Variation (TV) and integral priors to the formulation leading to considerable performance tested on signal restoration tasks such image denoising and video enhancement, and at the same time benefiting from its small computational cost., This work has been partially supported by the Spanish Ministry of Science and Innovation under the project DeeLight PID2020-117142GB-I00., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Sign language video retrieval with free-form textual queries

  • Cardoso Duarte, Amanda
  • Albanie, Samuel
  • Giró Nieto, Xavier
  • Varol, Gül
Systems that can efficiently search collections of sign language videos have been highlighted as a useful application of sign language technology. However, the problem of searching videos beyond individual keywords has received limited attention in the literature. To address this gap, in this work we introduce the task of sign language retrieval with textual queries: given a written query (e.g. a sentence) and a large collection of sign language videos, the objective is to find the signing video that best matches the written query. We propose to tackle this task by learning cross-modal embeddings on the recently introduced large-scale How2Sign dataset of American Sign Language (ASL). We identify that a key bottleneck in the performance of the system is the quality of the sign video embedding which suffers from a scarcity of labelled training data. We, therefore, propose SPOT-ALIGN, a framework for interleaving iterative rounds of sign spotting and feature alignment to expand the scope and scale of available training data. We validate the effectiveness of SPOT-ALIGN for learning a robust sign video embedding through improvements in both sign recognition and the proposed video retrieval task., This work was supported by the project PID2020-117142GB-I00, funded by MCIN/ AEI /10.13039/501100011033, ANR project CorVis ANR-21-CE23-0003- 01, and gifts from Google and Adobe. AD received support from la Caixa Foundation (ID 100010434), fellowship code LCF/BQ/IN18/11660029., Peer Reviewed, Objectius de Desenvolupament Sostenible::10 - Reducció de les Desigualtats, Objectius de Desenvolupament Sostenible::10 - Reducció de les Desigualtats::10.2 - Per a 2030, potenciar i promoure la inclusió social, econòmica i política de totes les persones, independentment de l’edat, sexe, discapacitat, raça, ètnia, origen, religió, situació econòmica o altra condició, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Real-time lane classification and accident detection for safer micromobility

  • Morros Rubió, Josep Ramon
  • Broquetas Ibars, Antoni
  • Mateo i Remacha, Alex
  • Puig Rabat, Jordi
  • Davins Jovells, Manel
The lack of knowledge of the micromobility regulations by e-scooter users is an important factor behind some of the accidents involving these vehicles. We present two modules that can increase the safety of the users and pedestrians: First, a computer vision model that analyses the video feed captured with a smartphone attached to the e-scooter, and predicts in real-time the type of lane in which the user is riding. This knowledge is used by an application which combines this information with GSNN location information and a database of mobility regulations, and informs the user when he/she is not complying with these regulations. Second, an accident detection system, using the smartphone accelerometer, that detects if there is a fall during the riding, so that the app can contact the authorities to determine the appropriate response. The experimental results show excellent results for both modules., We sincerely acknowledge the financial support that funding bodies have provided for this work. We thank the Spanish Ministry of Science, Innovation and Universities (PID2020-117142GB-I00 [DeeLight project] by MCIN/AEI/10.13039/501100011033). Additionally, we extend our gratitude to the EIT Urban Mobility, an initiative of the European Institute of Innovation and Technology (EIT), a body of the European Union, for their support under the RideSafeUM project, Peer Reviewed, Objectius de Desenvolupament Sostenible::11 - Ciutats i Comunitats Sostenibles, Objectius de Desenvolupament Sostenible::9 - Indústria, Innovació i Infraestructura, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Channel redundancy and overlap in convolutional neural networks with channel-wise NNK graphs

  • Bonet Solé, David
  • Ortega, Antonio
  • Ruiz Hidalgo, Javier
  • Shekkizhar, Sarath
Feature spaces in the deep layers of convolutional neural networks (CNNs) are often very high-dimensional and difficult to inter-pret. However, convolutional layers consist of multiple channels that are activated by different types of inputs, which suggests that more insights may be gained by studying the channels and how they relate to each other. In this paper, we first analyze theoretically channel-wise non-negative kernel (CW-NNK) regression graphs, which allow us to quantify the overlap between channels and, indirectly, the intrinsic dimension of the data representation manifold. We find that redundancy between channels is significant and varies with the layer depth and the level of regularization during training. Additionally, we observe that there is a correlation between channel overlap in the last convolutional layer and generalization performance. Our experimental results demonstrate that these techniques can lead to a better understanding of deep representations., This work was partially supported by the Spanish Research Agency (AEI) under project PID2020-117142GB-I00 of the call MCIN/ AEI /10.13039/501100011033, and by a DARPA grant (FA8750-19-2-1005) in Learning with Less Labels (LwLL) program and by NSF (CCF-2009032)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Enhancing Ki-67 cell segmentation with Dual U-Net models: a step towards uncertainty-informed active learning

  • Anglada Rotger, David
  • Sala Prat, Júlia
  • Marqués Acosta, Fernando
  • Salembier Clairon, Philippe Jean
  • Pardàs Feliu, Montse
The diagnosis and prognosis of breast cancer relies on histopathology image analysis where markers such as Ki-67 are increasingly important. The diagnosis using this marker is based on quantification of proliferation, which implies counting of Ki-67 positive and negative tumoral cells, excluding stromal cells. A common problem for automatic quantification of these images derives from overlapping and clustering of cells. We propose in this paper an automatic segmentation and classification system that overcomes this problem using two Convolutional Neural Networks (Dual U-Net), whose results are combined with a watershed algorithm. Taking into account that a major issue for the development of reliable neural networks is the availability of labeled databases, we also introduce an approach for epistemic uncertainty estimation that can be used for active learning in instance segmentation applications. We use Monte Carlo Dropout within our networks to quantify the model's confidence across its predictions, offering insights into areas of high uncertainty. Our results show how the postprocessed uncertainty maps can be used to refine ground truth annotations and to generate new labeled data with reduced annotation effort. To initialize the labeling and further reduce this effort, we propose a tool for groundtruth generation which is based on candidate generation with maxtree. Candidates are filtered based on extracted features which can be adjusted for the specific image typology, thereby facilitating precise model training and evaluation., This reseach has been funded by the project PID2020-116907RB-100 [AIMING] and PID2020-117142GB-I00 [DeeLight] by MCIN/ AEI /10.13039/ 501100011033., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

SkinningNet: two-stream graph convolutional neural network for skinning prediction of synthetic characters

  • Mosella Montoro, Albert
  • Ruiz Hidalgo, Javier
This work presents SkinningNet, an end-to-end Two-Stream Graph Neural Network architecture that computes skinning weights from an input mesh and its associated skeleton, without making any assumptions on shape class and structure of the provided mesh. Whereas previous methods pre-compute handcrafted features that relate the mesh and the skeleton or assume a fixed topology of the skeleton, the proposed method extracts this information in an end-to-end learnable fashion by jointly learning the best relationship between mesh vertices and skeleton joints. The proposed method exploits the benefits of the novel Multi-Aggregator Graph Convolution that combines the results of different aggregators during the summarizing step of the Message-Passing scheme, helping the operation to generalize for unseen topologies. Experimental results demonstrate the effectiveness of the contributions of our novel architecture, with SkinningNet outperforming current state-of-the-art alternatives., This work has been partially supported by the project PID2020-117142GB-I00, funded by MCIN/AEI /10.13039/501100011033., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Pixinwav: Residual steganography for hiding pixels in audio

  • Geleta Geleta, Margarita
  • Puntí Álvarez, Cristina
  • McGuinness, Kevin
  • Pons Puig, Jordi
  • Canton Ferrer, Cristian
  • Giró Nieto, Xavier
Steganography comprises the mechanics of hiding data in a host media that may be publicly available. While previous works focused on unimodal setups (e.g., hiding images in images, or hiding audio in audio), PixInWav targets the multimodal case of hiding images in audio. To this end, we propose a novel residual architecture operating on top of short-time discrete cosine transform (STDCT) audio spectrograms. Among our results, we find that the residual steganography setup we propose allows an encoding of the hidden image that is independent from the host audio without compromising quality. Accordingly, while previous works require both host and hidden signals to hide a signal, PixInWav can encode images offline—which can be later hidden, in a residual fashion, into any audio signal., Work partially supported by the European Union through the Erasmus+ student mobility program, Science Foundation Ireland (SFI) under grant numbers SFI/15/SIRG/3283 and SFI/12/RC/2289 P2, and the Spanish Research Agency (AEI) under project PID2020117142GB-I00 of the call MCIN/ AEI /10.13039/501100011033., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

H3D-Net: Few-shot high-fidelity 3D head reconstruction

  • Ramon Maldonado, Eduard
  • Triginer Garcés, Gil
  • Escurt i Gelabert, Janna
  • Pumarola Peris, Albert
  • García Giráldez, Jaime
  • Giró Nieto, Xavier
  • Moreno-Noguer, Francesc
Recent learning approaches that implicitly represent surface geometry using coordinate-based neural representations have shown impressive results in the problem of multi-view 3D reconstruction. The effectiveness of these techniques is, however, subject to the availability of a large number (several tens) of input views of the scene, and computationally demanding optimizations. In this paper, we tackle these limitations for the specific problem of few-shot full 3D head reconstruction, by endowing coordinate-based representations with a probabilistic shape prior that enables faster convergence and better generalization when using few input images (down to three). First, we learn a shape model of 3D heads from thousands of incomplete raw scans using implicit representations. At test time, we jointly overfit two coordinate-based neural networks to the scene, one modeling the geometry and another estimating the surface radiance, using implicit differentiable rendering. We devise a two-stage optimization strategy in which the learned prior is used to initialize and constrain the geometry during an initial optimization phase. Then, the prior is unfrozen and fine-tuned to the scene. By doing this, we achieve high-fidelity head reconstructions, including hair and shoulders, and with a high level of detail that consistently outperforms both state-of-the-art 3D Morphable Models methods in the few-shot scenario, and non-parametric methods when large sets of views are available., This work has been partially funded by the Spanish government with the projects MoHuCo PID2020-120049RBI00, DeeLight PID2020-117142GB-I00 and Maria de Maeztu Seal of Excellence MDM-2016-0656, and by the Government of Catalonia under 2017 DI 028., Peer Reviewed, Postprint (author's final draft)