Pasar al contenido principal

PROTECCION DE CULTIVOS DE PRECISION PARA CONSEGUIR OBJETIVOS DEL PACTO VERDE EUROPEO EN USO EFICIENTE Y REDUCCION DE FITOSANITARIOS MEDIANTE AGRICULTURA DE PRECISION

PID2021-126648OB-I00

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal para Impulsar la Investigación Científico-Técnica y su Transferencia
Subprograma Subprograma Estatal de Generación de Conocimiento
Convocatoria Proyectos de I+D+i (Generación de Conocimiento y Retos Investigación)
Año convocatoria 2021
Plan Estatal 2021-2023 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario UNIVERSIDAD DE LLEIDA

Resultados relacionados

Resultados totales (Incluyendo duplicados): 41
Encontrada(s) 1 página(s)

Repositori Obert UdL
Publicación Artículo científico (JournalArticle)

AmodalAppleSize_RGB-D dataset: RGB-D images of apple trees annotated with modal and amodal segmentation masks for fruit detection, visibility and size estimation

  • Gené Mola, Jordi
  • Ferrer Ferrer, Mar
  • Hemming, Jochen
  • Dalfsen, Pieter van
  • Hoog, Dirk de
  • Sanz Cortiella, Ricardo
  • Rosell Polo, Joan Ramon
  • Vilaplana Besler, Verónica
  • Ruiz Hidalgo, Javier
  • Gregorio López, Eduard
The present dataset comprises a collection of RGB-D apple tree images that can be used to train and test computer vision-based fruit detection and sizing methods. This dataset encompasses two distinct sets of data obtained from a Fuji and an Elstar apple orchards. The Fuji apple orchard sub-set consists of 3925 RGB-D images containing a total of 15,335 apples annotated with both modal and amodal apple segmentation masks. Modal masks denote the visible portions of the apples, whereas amodal masks encompass both visible and occluded apple regions. Notably, this dataset is the first public resource to incorporate on-tree fruit amodal masks. This pioneering inclusion addresses a critical gap in existing datasets, enabling the development of robust automatic fruit sizing methods and accurate fruit visibility estimation, particularly in the presence of partial occlusions. Besides the fruit segmentation masks, the dataset also includes the fruit size (calliper) ground truth for each annotated apple. The second sub-set comprises 2731 RGB-D images capturing five Elstar apple trees at four distinct growth stages. This sub-set includes mean diameter information for each tree at every growth stage and serves as a valuable resource for evaluating fruit sizing methods trained with the first sub-set. The present data was employed in the research paper titled “Looking behind occlusions: a study on amodal segmentation for robust on-tree apple fruit size estimation”., This work was partly funded by the Departament de Recerca i Universitats de la Generalitat de Catalunya (grant 2021 LLAV 00088 ), the Spanish Ministry of Science, Innovation and Universities (grants RTI2018-094222-B-I00 [PAgFRUIT project], PID2021-126648OB-I00 [PAgPROTECT project] and PID2020-117142GB-I00 [DeeLight project] by MCIN/AEI/10.13039/501100011033 and by “ERDF, a way of making Europe”, by the European Union). Data presented in this paper is also part of a Public Private Partnership project Precisie Tuinbouw, WP Fruit 4.0 (PPS KV 1604-025) and financed by Topsector Tuinbouw & Uitgangsmateriaal and various private companies. The work of Jordi Gené Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU. We would also like to thank Nufri (especially Santiago Salamero and Oriol Morreres) for their support during data acquisition.
Repositori Obert UdL
Publicación Artículo científico (JournalArticle)

Mobile terrestrial laser scanner vs. UAV photogrammetry to estimate woody crop canopy parameters - Part 2: Comparison for different crops and training systems

  • Torres-Sánchez, Jorge
  • Escolà i Agustí, Alexandre
  • de Castro, Ana I.
  • López-Granados, Francisca
  • Rosell Polo, Joan Ramon
  • Sebé Feixas, Francesc
  • Jiménez-Brenes, Francisco M.
  • Sanz Cortiella, Ricardo
  • Gregorio López, Eduard
  • Peña, José M.
The measurement of geometric canopy parameters in woody crops is an important task in Precision Agriculture because of their correlation with crop condition and productivity. In recent years, several technological approaches have been developed as an alternative to manual measurements, which are time- and labour-consuming. Two of the most commonly used 3D canopy characterization technologies are mobile terrestrial laser scanning (MTLS) based on light detection and ranging (LiDAR) sensors, and digital aerial photogrammetry (DAP) using imagery from uncrewed aerial vehicles (UAVs). Although both are state-of-the-art and have been fully tested and validated, a complete comparison between their geometric canopy parameter estimations in different woody crops and training systems has not been carried out. For this reason, a set of geometric parameters (canopy height, projected area, and volume) of a vineyard, an intensive peach orchard, and an intensive pear orchard were measured using UAV-DAP and MTLS-LiDAR. A comparison between both kinds of measurements was performed, accounting for the length of the sections in which the crop hedgerows were divided to extract the geometric parameters. Measurements from the UAV and the MTLS were highly correlated (R2 from 0.82 to 0.94) when considering the data from the three crops together, and the correlations were higher when analysing longer row sections. The canopy geometric parameters estimated using the MTLS-LiDAR always had higher values than those from the UAV-DAP. The results presented in this work provide useful data for a more informed selection of technological approaches for 3D crop characterization in Precision Fruticulture and high-throughput phenotyping., This research was funded by the projects DECIMAL (PID2020-113229RB-C41), PRODIGIA (PID2020-113229RB-C44), AGVANCE (AGL2013-48297-C2-2-R), and PAgPROTECT (PID2021-126648OB-I00), from the Spanish Ministry of Science and Innovation (MCIN/AEI/10.13039/501100011033). The authors also wish to thank Codorníu SA, celler Raimat and IRTA for having allowed the use of their vineyards, pear and peach orchards to conduct the trials described in this work.
Repositori Obert UdL
Publicación Artículo científico (JournalArticle)

Leafiness-LiDAR index and NDVI for identification of temporal patterns in super-intensive almond orchards as response to different management strategies

  • Sandonís Pozo, Leire
  • Oger, Baptiste
  • Tisseyre, Bruno
  • Llorens Calveras, Jordi
  • Escolà i Agustí, Alexandre
  • Pascual Roca, Miquel
  • Martínez Casasnovas, José Antonio
The use of super-intensive orchards is a growing trend in fruit production. The present study aims to improve management of these cropping systems by focusing on how agronomic decisions impact orchard dynamics in the short to medium term and by providing a decision-support approach based on stable temporal patterns from previous seasons. A multitemporal study using remote sensing and LiDAR was conducted in a commercial almond orchard over four growing seasons (2019-2022) to determine the optimal timing of image acquisition for variable pre-harvest treatments. A model-based clustering (mclust) was applied to optimal Sentinel-2 NDVI maps and apparent soil electrical conductivity (ECa) data, interpolated to the pixel centroids of Sentinel-2 image grids, to delineate potential management zones (PMZs). The leafiness-LiDAR index (LLI), a leaf area index (LAI) estimator, was obtained as ground truth after summer pruning and before harvesting, showing a significant influence of fertigation and pruning on the LAI, with summer pruning particularly influencing orchard dynamics. The optimal time for NDVI mapping was found to be two months after summer pruning in productive years and two weeks after in unproductive years. The delineated PMZs were consistent across seasons and corresponded to significant LAI differences. This method could contribute to improving resource management and sustainability in super-intensive commercial orchards., This research was funded by Grant RTI2018–094222-B-I00 (PAgFRUIT project) and PID2021–126648OB-I00 (PAgPROTECT project) by MCIN/AEI/10.13039/501100011033 and by “ERDF, a way of making Europe” of the European Union. The authors wish to thank Alrasa Agraria S.L. (Raimat, Lleida), the Centre for Technological and Industrial Development (Ministry of Science and Innovation, Government of Spain) and EuroChem Iberia S.A. for providing funds and experimental facilities, and to the members of the Research Group in AgroICT and Precision Agriculture (GRAP) of the Universitat de Lleida for their collaboration in data acquisition.
Repositori Obert UdL
Publicación Artículo científico (JournalArticle)

AKFruitYield: Modular benchmarking and video analysis software for Azure Kinect cameras for fruit size and fruit yield estimation in apple orchards

  • Miranda, Juan Carlos
  • Arnó Satorra, Jaume
  • Gené Mola, Jordi
  • Fountas, Spyros
  • Gregorio López, Eduard
AKFruitYield is a modular software that allows orchard data from RGB-D Azure Kinect cameras to be processed for fruit size and fruit yield estimation. Specifically, two modules have been developed: i) AK_SW_BENCHMARKER that makes it possible to apply different sizing algorithms and allometric yield prediction models to manually labelled color and depth tree images; and ii) AK_VIDEO_ANALYSER that analyses videos on which to automatically detect apples, estimate their size and predict yield at the plot or per hectare scale using the appropriate algorithms. Both modules have easy-to-use graphical interfaces and provide reports that can subsequently be used by other analysis tools., This work was partly funded by the Department of Research and Universities of the Generalitat de Catalunya (grants 2017 SGR 646) and by the Spanish Ministry of Science and Innovation/AEI/10.13039/501100011033/ERDF (grant RTI2018–094222-B-I00 [PAgFRUIT project] and PID2021–126648OB-I00 [PAgPROTECT project]). The Secretariat of Universities and Research of the Department of Business and Knowledge of the Generalitat de Catalunya and European Social Fund (ESF) are also thanked for financing Juan Carlos Miranda's pre-doctoral fellowship (2020 FI_B 00586). The work of Jordi Gené-Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU. The authors would also like to thank the Institut de Recerca i Tecnologia Agroalimentàries (IRTA) for allowing the use of their experimental fields, and in particular Dr. Luís Asín and Dr. Jaume Lordán who have contributed to the success of this work.
Repositori Obert UdL
Publicación Contribución de congreso (ConferenceOutput)

Análisis de los efectos de la sequía en manzanos a través del índice Leafiness-LiDAR

  • Sandonís Pozo, Leire
  • Martínez Casasnovas, José Antonio
  • Pascual Roca, Miquel
La sequía es un factor ambiental significativo que puede afectar de manera considerable el desarrollo y la productividad los cultivos frutícolas. En manzanos (Malus sylvestris L.Mill.), diversos autores han demostrado que la productividad y la calidad de los frutos está directamente relacionada con la cantidad de radiación solar absorbida por las copas. La gestión adecuada del dosel foliar durante períodos de sequía es esencial para minimizar sus consecuencias sobre la productividad de los manzanos. En este sentido, los sensores LiDAR (Light Detection And Ranging), utilizados en Fruticultura de Precisión para la caracterización 3D de la vegetación, pueden aportar la información necesaria para cuantificar las características geométricas y estructurales del dosel foliar de las plantaciones. Concretamente, en el presente trabajo se ha utilizado el índice Leafiness-LiDAR Index (LLI) medido en una plantación comercial intensiva de manzanos (Malus sylvestris L.Mill.) cv UEB 3264/2 Opal®, en la que se diseñó un experimento con diferentes dosis regadío a fin de simular las condiciones de sequía (desde condiciones deficitarias en diferentes períodos hasta riego completo). Los resultados obtenidos del análisis LiDAR permiten estimar adecuadamente el índice de área foliar (LAI) de los árboles, pudiendo relacionar estos indicadores con diversos e importantes atributos cuantitativos: crecimiento vegetativo, frondosidad del seto y rendimiento., Este trabajo es fruto del proyecto PID2021-126648OB-I00 (PAgPROTECT) concedido por MCIN/AEI/10.13039/501100011033 y por “ERDF, a way of make Europe”, de la Unión Europea.
Repositori Obert UdL
Publicación Contribución de congreso (ConferenceOutput)

Amodal segmentation for on-tree apple fruit size es timation with RGB-D images

  • Gené Mola, Jordi
  • Gregorio López, Eduard
  • Ferrer Ferrer, Mar
  • Blok, Pieter
  • Hemming, Jochen
  • Morros Rubió, Josep Ramon
  • Rosell Polo, Joan Ramon
  • Vilaplana Besler, Verónica
  • Ruiz Hidalgo, Javier
The detection and sizing of fruits with computer vision methods is of interest because it provides relevant information to improve the management of orchard farming. However, the presence of partially occluded fruits limits the performance of existing methods, making reliable fruit sizing a challenging task. While previous fruit segmentation works limit segmentation to the visible region of fruits (known as modal segmentation), in this work we propose an amodal segmentation algorithm to predict the complete shape, which includes its visible and occluded regions. CONCLUSIONS The main advantages of the present methodology are its robustness for measuring partially occluded fruits and the capability to determine the visibility percentage. Future works should evaluate the performance of the method with commercial RGB-D sensors, which would facilitate data collection., This work was partly funded by the Departament de Recerca i Universitats de la Generalitat de Catalunya (grant 2021 LLAV 00088), and the Spanish Minis try of Science, Innovation and Universities (grants PID2021-126648OB-I00 [PAgPROTECT] and PID2020-117142GB-I00 [DeeLight] by MCIN/ AEI/10.13039/501100011033 and by “ERDF, a way of making Europe”, by the European Union).
Repositori Obert UdL
Publicación Artículo científico (JournalArticle)

A methodology for the realistic assessment of 3D point clouds of fruit trees in full 3D context

  • Lavaquiol Colell, Bernat
  • Escolà i Agustí, Alexandre
  • Sanz Cortiella, Ricardo
  • Arnó Satorra, Jaume
  • Gené Mola, Jordi
  • Gregorio López, Eduard
  • Rosell Polo, Joan Ramon
  • Ninot, Jérome
  • Llorens Calveras, Jordi
The aim of this paper is to address the lack of standard methodologies for the assessment of 3D point clouds. We present a methodology to realistically assess the accuracy of 3D point clouds, enabling the evaluation in a full 3D context rather than based on isolated points. Additionally, it introduces three significant innovations: a) it bridges the gap related to the unknown error of the reference ground-truth point cloud; b) it provides separate metrics for location error and reconstruction error; and c) it introduces a procedure to compute the location error that eliminates the bias in the selection of point-pair picking between the DGT points and their corresponding pairs in the point cloud being assessed. The geometry and structure of trees are related to the vegetative parameters and productivity in fruit orchards. In consequence, obtaining a precise and accurate geometric characterization of canopies is of interest for implementing site-specific management strategies that optimize input rates and minimize the costs and environmental risks of agricultural operations. Among the different sensing technologies, sensors based on the principle of light detection and ranging (LiDAR) have emerged as the primary choice for accurate geometric characterization of orchards. However, to make informed orchard management decisions based on LiDAR-derived geometric and structural data, it is essential to assess the accuracy of LiDAR-based scanning systems. Unfortunately, there is currently a lack of standard methodologies to evaluate the accuracy of LiDAR-based systems in agricultural environments. This research paper presents a novel methodology to assess the location error and the reconstruction error of 3D point clouds in full 3D context. The methodology involves comparing LiDAR-derived point clouds to an accurate high-resolution 3D digital ground truth (DGT) obtained using digital photogrammetric techniques. One of the main difficulties when using a reference point cloud to assess point cloud errors is the selection of the points to be compared so that they can be considered as corresponding point pairs. When developing the methodology, four procedures of point pair selection and distance calculation were compared. The best performing procedure was selected and proposed as a standard for accuracy assessment of 3D point clouds. The proposed procedure minimizes the error attributed to the selection of the corresponding point pairs between the assessed point cloud and the reference DGT point cloud. Subsequently, the proposed methodology was tested and validated by assessing the accuracy of 46 different point clouds. The conclusions regarding the accuracy, applicability, and practical utility of the proposed methodology are supported by the determination of reconstruction errors and location errors in 46 point clouds obtained with the 3 different MTLS systems operated with different settings. The proposed methodology will be very useful for scanning system manufacturers, researchers, advisors and eventually advanced farmers to quantify the errors committed when characterizing tree canopies. This is crucial to enable accurate management operations in the framework of Precision Agriculture based on canopy variability. Furthermore, the methodology is expected to facilitate the design of new applications requiring high accuracy to be implemented in the near future., The present study is part of the PAgPROTECT project PID2021- 126648OB-I00 funded by the Spanish Ministry of Science and Innovation/AEI/10.13039/501100011033/FEDER, UE. This work was also partly funded by the Spanish Ministry of Science and Innovation/AEI/ 10.13039/501100011033/European Union NextGeneration/PRTR (grant TED2021-131871B-I00 [DIGIFRUIT project]).
Repositori Obert UdL
Publicación Artículo científico (JournalArticle)

LiDAR-derived indices and their relationship with productivity and oil quality attributes in high-density olive orchards

  • Sandonís Pozo, Leire
  • Rufat i Lamarca, Josep
  • Pascual Roca, Miquel
  • Villar Mir, Josep Ma.
  • Arnó Satorra, Jaume
  • Escolà i Agustí, Alexandre
  • Rosell Polo, Joan Ramon
  • Martínez Casasnovas, José Antonio
Optimising the leaf area index (LAI) and light exposure are key aspects to improve yield and quality in highdensity olive orchards. Traditionally, these characteristics are estimated using ceptometers, but this method is both labour intensive and dependent on light conditions, with limitations in effectiveness for precision horticulture management purposes. This study investigates the potential of four LiDAR-derived indices to: i) estimate the LAI and canopy light interception under different fertigation treatments, and ii) to assess their impact on yield and oil quality throughout a growing season in a high-density olive orchard. The treatments included two nitrogen dose rates (N0: 0 kg ha-¹ and N50: 50 kg ha-¹) combined with full irrigation (FI) and deficit irrigation (DI). The results provide strong evidence for the effectiveness of these indices in monitoring canopy response to treatments over time. The N50 and FI treatments increased the LAI and yield but led to poorer light attenuation within the canopy and lower oil quality. In contrast, N0 and DI promoted more light-permeable canopies and improved oil quality, with higher phenolic content, better oxidative stability and improved organoleptic oil attributes. These indices can improve the understanding of canopy light environments, help to optimise management practices, and ultimately improve the sustainability and profitability of intensive olive cultivation., This work was carried out with data acquired in the project IRTA2012–00059-C02–01 funded by Spain’s National Institute for Agricultural and Food Research and Technology, as well as in the projects AGL2010–22304-C04–03 and AGL2013–48297-C2–2-R funded by the Spanish Ministry of Economy and Competitiveness. The study was carried out under the framework of the grants RTI2018–094222-B-I00 (PAgFRUIT project) and PID2021–126648OB-I00 (PAgPROTECT project) through MCIN/AEI/10.13039/501100011033 and the European Union’s “ERDF, a way of making Europe”.
Repositori Obert UdL
Publicación Artículo científico (JournalArticle)

Parameter configuration to maximize accuracy in point clouds acquired with lidar-based terrestrial mobile laser scanners in fruit tree orchards

  • Lavaquiol Colell, Bernat
  • Escolà i Agustí, Alexandre
  • Arnó Satorra, Jaume
  • Grau, Joel
  • Ninot, Jérôme
  • Gómez, David
  • Llorens Calveras, Jordi
The aim of this paper is to determine the optimal parameter configuration to minimize LiDAR point cloud errors and maximize accuracy for agronomic applications in field conditions in the framework of Precision Agriculture. The structure and geometry of trees are closely linked to vegetative parameters and productivity in fruit orchards. Therefore, achieving precise and accurate geometric characterization of tree canopies is fundamental for developing site-specific management strategies for resource optimization while reducing costs and environmental impacts associated with agricultural practices. Among the various sensing technologies available, LiDAR-based sensors have emerged as one of the preferred options for obtaining accurate geometric data in orchards. For this reason, the paper focuses on analyzing how different parameter configurations used during point cloud production contribute to reconstruction and location errors. Specifically, it examines the influence of type of SLAM algorithm (triple resolution or SLAM3 and feature or SLAMF), the GNSS constellations in use, orbit and clock products, trajectory correction (PPK or TC) and navigation constraint resolution, and the distance to the GNSS base station. Regarding the reconstruction error, since the maximum difference between the best configuration (featuring SLAM3 and PPK) and the worst (featuring SLAMF and TC) is just 0.006 m, all parameter configurations are capable of efficiently reconstructing point clouds. Location error analysis showed that GPS-based solutions consistently exceeded 0.1 m, while GNSS-based solutions - involving any combination of two or more constellations among GPS, GLONASS, Galileo, and BeiDou - remained below 0.07 m. The best configurations include SLAM3 paired with PPK, regardless of the navigation constraint resolution, achieving an error of 0.032 m in the most favorable case. Although PPK trajectories performed better in GNSS-based solutions, they were less effective than TC when only GPS-based solutions were available. Additionally, using PPK trajectories ensures better time efficiency, as they are a prerequisite for generating TC trajectories. SLAM3 outperformed SLAMF under the tested conditions. This study represents a significant advancement in the optimization of LiDAR-derived point clouds generation. By identifying and fine-tuning key parameter configurations, it effectively minimizes both reconstruction and location errors, while simultaneously enhancing the temporal efficiency of the point cloud production process., The present study is part of the Project PAgPROTECT (PID2021–126648OB-I00) funded by MICIU/AEI /10.13039/501100011033 and by FEDER/ERDF, EU.
Repositori Obert UdL
Publicación Artículo científico (JournalArticle)

Fruit sizing using AI: A review of methods and challenges

  • Miranda, Juan Carlos
  • Gené Mola, Jordi
  • Zude-Sasse, Manuela
  • Tsoulias, Nikos
  • Escolà i Agustí, Alexandre
  • Arnó Satorra, Jaume
  • Rosell Polo, Joan Ramon
  • Sanz Cortiella, Ricardo
  • Martínez Casasnovas, José Antonio
  • Gregorio López, Eduard
Fruit size at harvest is an economically important variable for high-quality table fruit production in orchards and vineyards. In addition, knowing the number and size of the fruit on the tree is essential in the framework of precise production, harvest, and postharvest management. A prerequisite for analysis of fruit in a real-world environment is the detection and segmentation from background signal. In the last five years, deep learning convolutional neural network have become the standard method for automatic fruit detection, achieving F1-scores higher than 90 %, as well as real-time processing speeds. At the same time, different methods have been developed for, mainly, fruit size and, more rarely, fruit maturity estimation from 2D images and 3D point clouds. These sizing methods are focused on a few species like grape, apple, citrus, and mango, resulting in mean absolute error values of less than 4 mm in apple fruit. This review provides an overview of the most recent methodologies developed for in-field fruit detection/counting and sizing as well as few upcoming examples of maturity estimation. Challenges, such as sensor fusion, highly varying lighting conditions, occlusions in the canopy, shortage of public fruit datasets, and opportunities for research transfer, are discussed., his work was partly funded by the Department of Research and Universities of the Generalitat de Catalunya (grants 2017 SGR 646 and 2021 LLAV 00088) and by the Spanish Ministry of Science and Innovation / AEI/10.13039/501100011033 / FEDER (grants RTI2018-094222-B-I00 [PAgFRUIT project] and PID2021-126648OB-I00 [PAgPROTECT project]). The Secretariat of Universities and Research of the Department of Business and Knowledge of the Generalitat de Catalunya and European Social Fund (ESF) are also thanked for financing Juan Carlos Miranda’s pre-doctoral fellowship (2020 FI_B 00586). The work of Jordi Gené-Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU.
Repositori Obert UdL
Publicación Artículo científico (JournalArticle)

Video-based fruit detection and tracking: effects of scanning conditions on fruit load estimation

  • Felip-Pomés, , Marc
  • Gené Mola, Jordi
  • Arnó Satorra, Jaume
  • Lordan Sanahuja, Jaume
  • Ruiz Hidalgo, Javier
  • Morros Rubió, Josep Ramon
  • Arasanz-Malo,Miguel
  • Net Barnés, Francesc
  • Gregorio López, Eduard
Automated fruit counting and yield estimation systems are necessary for efficient orchard management. This study presents a computer vision system based on video multi-object-tracking for fruit load estimation in apple orchards and provides a comprehensive analysis of the system performance under diverse scanning conditions. The system integrates fruit detection, tracking, localization within orchard, and fruit load map generation. Experiments were carried out in an experimental apple orchard containing 420 apple trees. Data was collected with two different RGB-D sensors (Azure Kinect DK and ZED 2) at three different scanning distances (125 cm, 175 cm, and 225 cm) on two different dates prior to the harvest. Comparing the two evaluated sensors, Azure Kinect provided more consistent performance across different dates. Results also show that the longer scanning distance improves accuracy due to seeing the full tree view gives better fruit counts than close partial views. Between the two dates, best results were achieved near harvest due to fruit color at this stage, achieving a Mean Absolute Percentage Error (MAPE) of 6.91 % and a determination coefficient (R2) of 0.733 (using ZED2 sensor at 225 cm distance). Finally, a test comparing scanning from one or both sides of the tree row showed that bilateral scanning improved fruit load estimation at the stretch level by incorporating information from both sides of the canopy. The results of this work demonstrate the effectiveness of the video fruit tracking systems as a useful tool for automating fruit load estimation., This work was partly funded by the Department of Research and Universities of the Generalitat de Catalunya (grants 2017, SGR 646 and 2021 LLAV 00088), by the Spanish Ministry of Science and Innovation /AEI/10.13039/501100011033 / ERDF (grants RTI2018-094222-B-I00 [PAgFRUIT project], PID2021-126648OB-I00 [PAgPROTECT project], PID2024-161868OB-I00 [C3DRUM project], and PID2024-156495OR-C31 [YIELD4CAST]), by the Spanish Ministry of Science and Innovation / AEI/10.13039/501100011033 / European Union NextGeneration / PRTR (grant TED2021-131871B-I00 [DIGIFRUIT project]), and by the Agrotecnio-IRTA Joint Projects Call 2025 (PhotonFruit project). The work of Jordi Gené-Mola was partially supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU.
Repositori Obert UdL
Publicación Artículo científico (JournalArticle)

Assessing automatic data processing algorithms for RGB-D cameras to predict fruit size and weight in apples

  • Miranda, Juan Carlos
  • Arnó Satorra, Jaume
  • Gené Mola, Jordi
  • Lordan Sanahuja, Jaume
  • Asin Jones, Luis
  • Gregorio López, Eduard
Data acquired using an RGB-D Azure Kinect DK camera were used to assess different automatic algorithms to estimate the size, and predict the weight of non-occluded and occluded apples. The programming of the algorithms included: (i) the extraction of images of regions of interest (ROI) using manual delimitation of bounding boxes or binary masks; (ii) estimating the lengths of the major and minor geometric axes for the purpose of apple sizing; and (iii) predicting the final weight by allometric modelling. In addition to the use of bounding boxes, the algorithms also allowed other post-mask settings (circles, ellipses and rotated rectangles) to be implemented, and different depth options (distance between the RGB-D camera and the fruits detected) for subsequent sizing through the application of the thin lens theory. Both linear and nonlinear allometric models demonstrated the ability to predict apple weight with a high degree of accuracy (R2 greater than 0.942 and RMSE < 16 g). With respect to non-occluded apples, the best weight predictions were achieved using a linear allometric model including both the major and minor axes of the apples as predictors. The mean absolute percentage error (MAPE) ranged from 5.1% to 5.7% with respective RMSE of 11.09 g and 13.02 g, depending to whether circles, ellipses, or bounding boxes were used to adjust fruit shape. The results were therefore promising and open up the possibility of implementing reliable in-field apple measurements in real time. Importantly, final weight prediction error and intermediate size estimation errors (from sizing algorithms) interact but in a way that is not easily quantifiable when weight allometric models with implicit prediction error are used. In addition, allometric models should be reviewed when applied to other apple cultivars, fruit development stages or even for different fruit growth conditions depending on canopy management., This work was partly funded by the Department of Research and Universities of the Generalitat de Catalunya (grants 2017, SGR 646 and 2021 LLAV 00088), by the Spanish Ministry of Science and Innovation / AEI/10.13039/501100011033 / ERDF (grants RTI2018-094222-B-I00 [PAgFRUIT project], PID2021-126648OB-I00 [PAgPROTECT project]) and by the Spanish Ministry of Science and Innovation / AEI/10.13039/ 501100011033 / European Union NextGeneration / PRTR (grantTED2021-131871B-I00 [DIGIFRUIT project]). We would also like to thank the Secretariat of Universities and Research of the Department of Business and Knowledge of the Generalitat de Catalunya and the European Social Fund (ESF) for financing Juan Carlos Miranda’s pre-doctoral fellowship (2020 FI_B 00586). The work of Jordi Gen´e-Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU.
Repositori Obert UdL
Publicación Artículo científico (JournalArticle)

Video-Based Fruit Detection and Tracking for Apple Counting and Mapping

  • Gené Mola, Jordi
  • Felip Pomés, Marc
  • Net Barnés, Francesc
  • Morros Rubió, Josep Ramon
  • Miranda, Juan Carlos
  • Arnó Satorra, Jaume
  • Asin Jones, Luis
  • Lordan Sanahuja, Jaume
  • Ruiz Hidalgo, Javier
  • Gregorio López, Eduard
Automatic fruit counting systems have garnered interest from farmers and agronomists to monitor fruit production, predict yields in advance, and identify production variability across orchards. However, accurately counting fruits poses challenges, particularly due to occlusions. This study proposes a multi-view sensing approach using continuous motion videos captured by a camera moved along the row of trees, followed by fruit detection in all video frames and application of Multi-Object Tracking (MOT) algorithms to prevent double-counting. Three tracking methods, namely SORT, DeepSORT, and ByteTrack, are compared for fruit counting using the YOLOv5x object detector. The methodology is applied to map fruit production in an experimental apple orchard at two different dates: four weeks and one week before harvest. The results demonstrate that ByteTrack (MOTA=0.682; IDF1=0.837; HOTA=0.689) outperforms SORT and DeepSORT, indicating its superior tracking performance. Computational efficiency analysis reveals similar processing times between SORT and ByteTrack (about 15 ms), while DeepSORT requires significantly more processing time per image (128 ms). Fruit counting evaluation shows reasonably accurate yield predictions on both dates, with reduced errors and improved performance closer to the harvest date (MAPE=7.47 %; R2=0.70). The system proves effective in estimating orchard fruit production using computer vision technology, offering valuable insights for yield forecasting. These findings contribute to optimizing fruit production and supporting precision agriculture practices. The code and the dataset have been made publicly available and a video visualization of results is accessible at http://www.grap.udl.cat/en/publications/video_fruit_counting., This work was partly funded by the Departament de Recerca i Universitats de la Generalitat de Catalunya (grant 2021 LLAV 00088), the Spanish Ministry of Science, Innovation and Universities (grants RTI2018-094222-B-I00 [PAgFRUIT project], PID2021-126648OB-I00 [PAgPROTECT project] and PID2020-117142GB-I00 [DeeLight project] by MCIN/AEI/10.13039/501100011033 and by “ERDF, a way of making Europe”, by the European Union). The work of Jordi Gené Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU. The Secretariat of Universities and Research of the Department of Business and Knowledge of the Generalitat de Catalunya and European Social Fund (ESF) are also thanked for financing Juan Carlos Miranda’s pre doctoral fellowship (2020 FI_B 00586).
Repositori Obert UdL
Publicación Artículo científico (JournalArticle)

Looking behind occlusions: A study on amodal segmentation for robust on-tree apple fruit size estimation

  • Gené Mola, Jordi
  • Ferrer Ferrer, Mar
  • Gregorio López, Eduard
  • Blok, Pieter
  • Hemming, Jochen
  • Morros Rubió, Josep Ramon
  • Rosell Polo, Joan Ramon
  • Vilaplana Besler, Verónica
  • Ruiz Hidalgo, Javier
The detection and sizing of fruits with computer vision methods is of interest because it provides relevant information to improve the management of orchard farming. However, the presence of partially occluded fruits limits the performance of existing methods, making reliable fruit sizing a challenging task. While previous fruit segmentation works limit segmentation to the visible region of fruits (known as modal segmentation), in this work we propose an amodal segmentation algorithm to predict the complete shape, which includes its visible and occluded regions. To do so, an end-to-end convolutional neural network (CNN) for sim ultaneous modal and amodal instance segmentation was implemented. The predicted amodal masks were used to estimate the fruit diameters in pixels. Modal masks were used to identify the visible region and measure the distance between the apples and the camera using the depth image. Finally, the fruit diameters in millimetres (mm) were computed by applying the pinhole camera model. The method was developed with a Fuji apple dataset consisting of 3925 RGB-D images acquired at different growth stages with a total of 15,335 annotated apples, and was subsequently tested in a case study to measure the diameter of Elstar apples at different growth stages. Fruit detection results showed an F1-score of 0.86 and the fruit diameter results reported a mean absolute error (MAE) of 4.5 mm and R2 = 0.80 irrespective of fruit visibility. Besides the diameter estimation, modal and amodal masks were used to automatically determine the percentage of visibility of measured apples. This feature was used as a confidence value, improving the diameter estimation to MAE = 2.93 mm and R2 = 0.91 when limiting the size estimation to fruits detected with a visibility higher than 60%. The main advantages of the present methodology are its robustness for measuring partially occluded fruits and the capability to determine the visibility percentage. The main limitation is that depth images were generated by means of photogrammetry methods, which limits the efficiency of data acquisition. To overcome this limitation, future works should consider the use of commercial RGB-D sensors. The code and the dataset used to evaluate the method have been made publicly available at https://github.com/GRAP-UdL-AT/Amodal_Fruit_Sizing., This work was partly funded by the Departament de Recerca i Universitats de la Generalitat de Catalunya (grant 2021 LLAV 00088), the Spanish Ministry of Science, Innovation and Universities (grants RTI2018-094222-B-I00 [PAgFRUIT project], PID2021-126648OB-I00 [PAgPROTECT project] and PID2020-117142GB-I00 [DeeLight project] by MCIN/AEI/10.13039/501100011033 and by “ERDF, a way of making Europe”, by the European Union). The work of Jordi Gen´e Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU. We would also like to thank Nufri (especially Santiago Salamero and Oriol Morreres) for their support during data acquisition, and Pieter van Dalfsen and Dirk de Hoog from Wageningen University & Research for additional data collection used in the case study.
Repositori Obert UdL
Publicación Contribución de congreso (ConferenceOutput)

Uncertainty analysis of a LiDAR-based MTLS point cloud using a high-resolution ground-truth

  • Lavaquiol Colell, Bernat
  • Llorens Calveras, Jordi
  • Sanz Cortiella, Ricardo
  • Arnó Satorra, Jaume
  • Escolà i Agustí, Alexandre
Poster Proceedings of the 14th European Conference on Precision Agriculture, 2-6 July 2023, Bologna, Italy, The study of plant geometry is crucial to design specific management by providing the optimal quantities of nutrients, fertilizers, pesticides and irrigation rates. Before the advent of the first 3D characterization systems, it was very laborious to obtain accurate commercial scale 3D crop data. Nowadays, there are sensing systems which allow 3D canopy characterization to be performed in a relatively simple and fast way. LiDAR (light detection and ranging) sensors have been widely used in agriculture. When 3D scanning techniques are used, it is essential to be aware of the total measurement error. One of the limitations when using real data is the absence of ground-truth (GT) to compare the obtained measurements . In a previous research [1], validated a high-resolution 3D point cloud on an actual defoliated tree obtained from RGB images and stereo-photogrammetry techniques. This accurate 3D point cloud can be used as digital ground-truth (DGT) to validate 3D LiDAR point. The accuracy of the scanning system includes the errors committed by the sensor, the positioning system (GNSS), the data acquisition set up, the point cloud generation algorithms and the georeferentiation of the DGT., The present study is part of the PAgPROTECT project PID2021-126648OB-I00 funded by the Spanish Ministry of Science and Innovation / AEI /10.13039/501100011033 / FEDER, UE.
Repositori Obert UdL
Publicación Artículo científico (JournalArticle)

Mobile terrestrial laser scanner vs. UAV photogrammetry to estimate woody crop canopy parameters - Part 1: Methodology and comparison in vineyards

  • Escolà i Agustí, Alexandre
  • Peña, José M.
  • López-Granados, Francisca
  • Rosell Polo, Joan Ramon
  • de Castro, Ana I.
  • Gregorio López, Eduard
  • Jiménez-Brenes, Francisco M.
  • Sanz Cortiella, Ricardo
  • Sebé Feixas, Francesc
  • Llorens Calveras, Jordi
  • Torres-Sánchez, Jorge
Characterizing crop canopies is especially important in the management of woody crops. In this article, two systems were compared to characterise a 50 m long vineyard row section. One of the systems was a mobile terrestrial laser scanner based on a light detection and ranging (LiDAR) sensor (MTLS-LiDAR). The other was an uncrewed aerial vehicle (UAV) based system using digital aerial photogrammetry (UAV-DAP). The resulting 3D point clouds were assessed qualitatively and quantitatively. Canopy heights, widths and volumes were obtained in 0.1 m long sections along the studied row. All the parameters derived from the two systems presented statistically significant differences. The coefficients of determination between systems were 0.619 for canopy maximum heights above ground level (agl), 0.686 for 90th percentile (P90) heights agl, and 0.283 and 0.274 for maximum and P90 vegetated heights, respectively. Coefficients of determination between averaged maximum canopy width and P90 canopy width were 0.328 and 0.317, respectively. Coefficients of determination between cross-sectional areas determined from maximum widths, P90 widths and from the occupancy grid method were 0.423, 0.409 and 0.334, respectively. Total canopy volume for the entire row obtained from the three cross section estimation methods differed between 19 m3 and 25 m3. The reasons found were that the MTLS-LiDAR-derived point cloud captured the canopy top and side variability but could be affected by occlusions, mixed pixels and tall grass-like weeds present in the surveyed area. For its part, the UAV-DAP-derived point cloud tended to miss top and side shoots and somewhat smoothed canopy variability. As neither of the systems is optimal, a balance needs to be found according to the specific requirements of the survey. For this purpose, a list of pros and cons is presented to support the selection of one of the two systems for canopy monitoring. The MTLS-LiDAR system should be chosen when high detail is required but small areas are to be scanned. Alternatively, the UAV-DAP system should be chosen when large areas are to be monitored and when canopy detail is not so important. Further results are presented in Part 2 for a larger area and including pear and peach orchards with different training systems. Future research is to be conducted on how the compared systems affect variability detection and support variable-rate prescriptions., This research was funded by the projects AGVANCE (AGL2013-48297-C2-2-R), PAgPROTECT (PID2021-126648OB-I00), DECIMAL (PID2020-113229RB-C41) and PRODIGIA (PID2020-113229RB-C44) from the Spanish Ministry of Science and Innovation MICIN/AEI/10.13039/501100011033/FEDER, UE. The authors also want to thank Codorníu SA and Celler Raimat for allowing the use of one of their vineyards to conduct the trial described in this work.
Repositori Obert UdL
Publicación Contribución de congreso (ConferenceOutput)

A new Leafiness-LiDAR index to estimate light interception in intensive olive orchards

  • Sandonís Pozo, Leire
  • Martínez Casasnovas, José Antonio
  • Escolà i Agustí, Alexandre
  • Rosell Polo, Joan Ramon
  • Rufat i Lamarca, Josep
  • Pascual Roca, Miquel
The 14th European Conference on Precision Agriculture, July 2-6, 2023, Bologna, Italy, Canopy light interception constitutes an important yield limiting factor in high density olive orchards. However, its characterisation still implies laborious measurements. A new index, the Leafiness-LiDAR index (LLI), is presented as a LAI estimator. LLI combines LiDAR-derived parameters: Cross-Section and Leafiness from 3D point clouds. To validate the results, photosynthetically active radiation (PAR) measurements, canopy volume, yield and quality parameters were collected and analysed. LLI showed significant correlations both with PAR and canopy volume (r = 0.8) and quality parameters (r = -0.6). LLI may be useful as an early decision canopy monitoring tool in the framework of Precision Fructiculture., This work was carried out with data acquired in the project IRTA2012-00059-C02-01 funded by the Instituto Nacional de Investigación y Tecnología Agraria y Alimentaria, as well as in the projects AGL2010-22304-C04-03 and AGL2013-48297-C2-2-R funded by the Spanish Ministry of Economy and Competitiveness and developed under the frame of the grants RTI2018-094222-B-I00 (PAgFRUIT project) and PID2021-126648OB-I00 (PAgPROTECT project) by MCIN/AEI/10.13039/501100011033 and by “ERDF, a way of making Europe”, by the European Union.
Repositori Obert UdL
Publicación Contribución de congreso (ConferenceOutput)

Low-cost terrestrial photogrammetry for orchard sidewards 3D reconstruction

  • Martínez Casasnovas, José Antonio
  • Rosell Tarragó, Miquel
  • Rosell Polo, Joan Ramon
  • Sanz Cortiella, Ricardo
  • Gregorio López, Eduard
  • Gené Mola, Jordi
  • Arnó Satorra, Jaume
  • Plata Moreno, José Manuel
  • Escolà i Agustí, Alexandre
The present study was part of the grants RTI2018-094222-B-I00 and PID2021-126648OB-I00 (PAgFRUIT and PAgPROTECT projects), funded by MCIN/AEI/10.13039/501100011033 and by “ERDF, a way of making Europe”, by the European Union
Repositori Obert UdL
Publicación Artículo científico (JournalArticle)

Simultaneous fruit detection and size estimation using multitask deep neural networks

  • Ferrer Ferrer, Mar
  • Ruiz Hidalgo, Javier
  • Gregorio López, Eduard
  • Vilaplana Besler, Verónica
  • Morros Rubió, Josep Ramon
  • Gené Mola, Jordi
The measurement of fruit size is of great interest to estimate the yield and predict the harvest resources in advance. This work proposes a novel technique for in-field apple detection and measurement based on Deep Neural Networks. The proposed framework was trained with RGB-D data and consists of an end-to-end multitask Deep Neural Network architecture specifically designed to perform the following tasks: 1) detection and segmentation of each fruit from its surroundings; 2) estimation of the diameter of each detected fruit. The methodology was tested with a total of 15,335 annotated apples at different growth stages, with diameters varying from 27 mm to 95 mm. Fruit detection results reported an F1-score for apple detection of 0.88 and a mean absolute error of diameter estimation of 5.64 mm. These are state-of-the-art results with the additional advantages of: a) using an end-to-end multitask trainable network; b) an efficient and fast inference speed; and c) being based on RGB-D data which can be acquired with affordable depth cameras. On the contrary, the main disadvantage is the need of annotating a large amount of data with fruit masks and diameter ground truth to train the model. Finally, a fruit visibility analysis showed an improvement in the prediction when limiting the measurement to apples above 65% of visibility (mean absolute error of 5.09 mm). This suggests that future works should develop a method for automatically identifying the most visible apples and discard the prediction of highly occluded fruits., This work was partly funded by the Departament de Recerca i Universitats de la Generalitat de Catalunya (grant 2021 LLAV 00088), the Spanish Ministry of Science, Innovation and Universities (grants RTI2018-094222-B-I00[PAgFRUIT project], PID2021-126648OB-I00 [PAgPROTECT project] and PID2020- 117142 GB-I00 [DeeLight project] by MCIN/AEI/10.13039/ 501100011033 and by “ERDF, a way of making Europe”, by the European Union). The work of Jordi Gene Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Mobile terrestrial laser scanner vs. UAV photogrammetry to estimate woody crop canopy parameters – Part 2: Comparison for different crops and training systems

  • Torres-Sánchez, Jorge
  • Escolà Agustí, Alexandre
  • Castro, Ana Isabel de
  • López Granados, Francisca
  • Rosell-Polo, Joan R.
  • Sebé, Francesc
  • Jiménez-Brenes, Francisco Manuel
  • Sanz, Ricardo
  • Gregorio, Eduard
  • Peña Barragán, José Manuel
10 Pág., The measurement of geometric canopy parameters in woody crops is an important task in Precision Agriculture because of their correlation with crop condition and productivity. In recent years, several technological approaches have been developed as an alternative to manual measurements, which are time- and labour-consuming. Two of the most commonly used 3D canopy characterization technologies are mobile terrestrial laser scanning (MTLS) based on light detection and ranging (LiDAR) sensors, and digital aerial photogrammetry (DAP) using imagery from uncrewed aerial vehicles (UAVs). Although both are state-of-the-art and have been fully tested and validated, a complete comparison between their geometric canopy parameter estimations in different woody crops and training systems has not been carried out. For this reason, a set of geometric parameters (canopy height, projected area, and volume) of a vineyard, an intensive peach orchard, and an intensive pear orchard were measured using UAV-DAP and MTLS-LiDAR. A comparison between both kinds of measurements was performed, accounting for the length of the sections in which the crop hedgerows were divided to extract the geometric parameters. Measurements from the UAV and the MTLS were highly correlated (R2 from 0.82 to 0.94) when considering the data from the three crops together, and the correlations were higher when analysing longer row sections. The canopy geometric parameters estimated using the MTLS-LiDAR always had higher values than those from the UAV-DAP. The results presented in this work provide useful data for a more informed selection of technological approaches for 3D crop characterization in Precision Fruticulture and high-throughput phenotyping., This research was funded by the projects DECIMAL (PID2020-113229RB-C41), PRODIGIA (PID2020-113229RB-C44), AGVANCE (AGL2013-48297-C2-2-R), and PAgPROTECT (PID2021-126648OB-I00), from the Spanish Ministry of Science and Innovation (MCIN/AEI/10.13039/501100011033). The authors also wish to thank Codorníu SA, celler Raimat and IRTA for having allowed the use of their vineyards, pear and peach orchards to conduct the trials described in this work., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Mobile terrestrial laser scanner vs. UAV photogrammetry to estimate woody crop canopy parameters – Part 1: Methodology and comparison in vineyards

  • Escolà Agustí, Alexandre
  • Peña Barragán, José Manuel
  • López Granados, Francisca
  • Rosell-Polo, Joan R.
  • Castro, Ana Isabel de
  • Gregorio, Eduard
  • Jiménez-Brenes, Francisco Manuel
  • Sanz, Ricardo
  • Sebé, Francesc
  • Llorens, Jordi
  • Torres-Sánchez, Jorge
© 2023 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/)., Characterizing crop canopies is especially important in the management of woody crops. In this article, two systems were compared to characterise a 50 m long vineyard row section. One of the systems was a mobile terrestrial laser scanner based on a light detection and ranging (LiDAR) sensor (MTLS-LiDAR). The other was an uncrewed aerial vehicle (UAV) based system using digital aerial photogrammetry (UAV-DAP). The resulting 3D point clouds were assessed qualitatively and quantitatively. Canopy heights, widths and volumes were obtained in 0.1 m long sections along the studied row. All the parameters derived from the two systems presented statistically significant differences. The coefficients of determination between systems were 0.619 for canopy maximum heights above ground level (agl), 0.686 for 90th percentile (P90) heights agl, and 0.283 and 0.274 for maximum and P90 vegetated heights, respectively. Coefficients of determination between averaged maximum canopy width and P90 canopy width were 0.328 and 0.317, respectively. Coefficients of determination between cross-sectional areas determined from maximum widths, P90 widths and from the occupancy grid method were 0.423, 0.409 and 0.334, respectively. Total canopy volume for the entire row obtained from the three cross section estimation methods differed between 19 m3 and 25 m3. The reasons found were that the MTLS-LiDAR-derived point cloud captured the canopy top and side variability but could be affected by occlusions, mixed pixels and tall grass-like weeds present in the surveyed area. For its part, the UAV-DAP-derived point cloud tended to miss top and side shoots and somewhat smoothed canopy variability. As neither of the systems is optimal, a balance needs to be found according to the specific requirements of the survey. For this purpose, a list of pros and cons is presented to support the selection of one of the two systems for canopy monitoring. The MTLS-LiDAR system should be chosen when high detail is required but small areas are to be scanned. Alternatively, the UAV-DAP system should be chosen when large areas are to be monitored and when canopy detail is not so important. Further results are presented in Part 2 for a larger area and including pear and peach orchards with different training systems. Future research is to be conducted on how the compared systems affect variability detection and support variable-rate prescriptions., This research was funded by the projects AGVANCE (AGL2013-48297-C2-2-R), PAgPROTECT (PID2021-126648OB-I00), DECIMAL (PID2020-113229RB-C41) and PRODIGIA (PID2020-113229RB-C44) from the Spanish Ministry of Science and Innovation MICIN/AEI/10.13039/501100011033/FEDER, UE., Peer reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Mobile terrestrial laser scanner vs. UAV photogrammetry to estimate woody crop canopy parameters - Part 1: Methodology and comparison in vineyards

  • Escolà i Agustí, Alexandre
  • Peña, José M.
  • López-Granados, Francisca
  • Rosell Polo, Joan Ramon
  • de Castro, Ana I.
  • Gregorio López, Eduard
  • Jiménez-Brenes, Francisco M.
  • Sanz Cortiella, Ricardo
  • Sebé Feixas, Francesc
  • Llorens Calveras, Jordi
  • Torres-Sánchez, Jorge
Characterizing crop canopies is especially important in the management of woody crops. In this article, two systems were compared to characterise a 50 m long vineyard row section. One of the systems was a mobile terrestrial laser scanner based on a light detection and ranging (LiDAR) sensor (MTLS-LiDAR). The other was an uncrewed aerial vehicle (UAV) based system using digital aerial photogrammetry (UAV-DAP). The resulting 3D point clouds were assessed qualitatively and quantitatively. Canopy heights, widths and volumes were obtained in 0.1 m long sections along the studied row. All the parameters derived from the two systems presented statistically significant differences. The coefficients of determination between systems were 0.619 for canopy maximum heights above ground level (agl), 0.686 for 90th percentile (P90) heights agl, and 0.283 and 0.274 for maximum and P90 vegetated heights, respectively. Coefficients of determination between averaged maximum canopy width and P90 canopy width were 0.328 and 0.317, respectively. Coefficients of determination between cross-sectional areas determined from maximum widths, P90 widths and from the occupancy grid method were 0.423, 0.409 and 0.334, respectively. Total canopy volume for the entire row obtained from the three cross section estimation methods differed between 19 m3 and 25 m3. The reasons found were that the MTLS-LiDAR-derived point cloud captured the canopy top and side variability but could be affected by occlusions, mixed pixels and tall grass-like weeds present in the surveyed area. For its part, the UAV-DAP-derived point cloud tended to miss top and side shoots and somewhat smoothed canopy variability. As neither of the systems is optimal, a balance needs to be found according to the specific requirements of the survey. For this purpose, a list of pros and cons is presented to support the selection of one of the two systems for canopy monitoring. The MTLS-LiDAR system should be chosen when high detail is required but small areas are to be scanned. Alternatively, the UAV-DAP system should be chosen when large areas are to be monitored and when canopy detail is not so important. Further results are presented in Part 2 for a larger area and including pear and peach orchards with different training systems. Future research is to be conducted on how the compared systems affect variability detection and support variable-rate prescriptions., This research was funded by the projects AGVANCE (AGL2013-48297-C2-2-R), PAgPROTECT (PID2021-126648OB-I00), DECIMAL (PID2020-113229RB-C41) and PRODIGIA (PID2020-113229RB-C44) from the Spanish Ministry of Science and Innovation MICIN/AEI/10.13039/501100011033/FEDER, UE. The authors also want to thank Codorníu SA and Celler Raimat for allowing the use of one of their vineyards to conduct the trial described in this work.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Amodal segmentation for on-tree apple fruit size es timation with RGB-D images

  • Gené Mola, Jordi
  • Gregorio López, Eduard
  • Ferrer Ferrer, Mar
  • Blok, Pieter
  • Hemming, Jochen
  • Morros Rubió, Josep Ramon
  • Rosell Polo, Joan Ramon
  • Vilaplana Besler, Verónica
  • Ruiz Hidalgo, Javier
The detection and sizing of fruits with computer vision methods is of interest because it provides relevant information to improve the management of orchard farming. However, the presence of partially occluded fruits limits the performance of existing methods, making reliable fruit sizing a challenging task. While previous fruit segmentation works limit segmentation to the visible region of fruits (known as modal segmentation), in this work we propose an amodal segmentation algorithm to predict the complete shape, which includes its visible and occluded regions. CONCLUSIONS The main advantages of the present methodology are its robustness for measuring partially occluded fruits and the capability to determine the visibility percentage. Future works should evaluate the performance of the method with commercial RGB-D sensors, which would facilitate data collection., This work was partly funded by the Departament de Recerca i Universitats de la Generalitat de Catalunya (grant 2021 LLAV 00088), and the Spanish Minis try of Science, Innovation and Universities (grants PID2021-126648OB-I00 [PAgPROTECT] and PID2020-117142GB-I00 [DeeLight] by MCIN/ AEI/10.13039/501100011033 and by “ERDF, a way of making Europe”, by the European Union).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

A methodology for the realistic assessment of 3D point clouds of fruit trees in full 3D context

  • Lavaquiol Colell, Bernat
  • Escolà i Agustí, Alexandre
  • Sanz Cortiella, Ricardo
  • Arnó Satorra, Jaume
  • Gené Mola, Jordi
  • Gregorio López, Eduard
  • Rosell Polo, Joan Ramon
  • Ninot, Jérome
  • Llorens Calveras, Jordi
The aim of this paper is to address the lack of standard methodologies for the assessment of 3D point clouds. We present a methodology to realistically assess the accuracy of 3D point clouds, enabling the evaluation in a full 3D context rather than based on isolated points. Additionally, it introduces three significant innovations: a) it bridges the gap related to the unknown error of the reference ground-truth point cloud; b) it provides separate metrics for location error and reconstruction error; and c) it introduces a procedure to compute the location error that eliminates the bias in the selection of point-pair picking between the DGT points and their corresponding pairs in the point cloud being assessed. The geometry and structure of trees are related to the vegetative parameters and productivity in fruit orchards. In consequence, obtaining a precise and accurate geometric characterization of canopies is of interest for implementing site-specific management strategies that optimize input rates and minimize the costs and environmental risks of agricultural operations. Among the different sensing technologies, sensors based on the principle of light detection and ranging (LiDAR) have emerged as the primary choice for accurate geometric characterization of orchards. However, to make informed orchard management decisions based on LiDAR-derived geometric and structural data, it is essential to assess the accuracy of LiDAR-based scanning systems. Unfortunately, there is currently a lack of standard methodologies to evaluate the accuracy of LiDAR-based systems in agricultural environments. This research paper presents a novel methodology to assess the location error and the reconstruction error of 3D point clouds in full 3D context. The methodology involves comparing LiDAR-derived point clouds to an accurate high-resolution 3D digital ground truth (DGT) obtained using digital photogrammetric techniques. One of the main difficulties when using a reference point cloud to assess point cloud errors is the selection of the points to be compared so that they can be considered as corresponding point pairs. When developing the methodology, four procedures of point pair selection and distance calculation were compared. The best performing procedure was selected and proposed as a standard for accuracy assessment of 3D point clouds. The proposed procedure minimizes the error attributed to the selection of the corresponding point pairs between the assessed point cloud and the reference DGT point cloud. Subsequently, the proposed methodology was tested and validated by assessing the accuracy of 46 different point clouds. The conclusions regarding the accuracy, applicability, and practical utility of the proposed methodology are supported by the determination of reconstruction errors and location errors in 46 point clouds obtained with the 3 different MTLS systems operated with different settings. The proposed methodology will be very useful for scanning system manufacturers, researchers, advisors and eventually advanced farmers to quantify the errors committed when characterizing tree canopies. This is crucial to enable accurate management operations in the framework of Precision Agriculture based on canopy variability. Furthermore, the methodology is expected to facilitate the design of new applications requiring high accuracy to be implemented in the near future., The present study is part of the PAgPROTECT project PID2021- 126648OB-I00 funded by the Spanish Ministry of Science and Innovation/AEI/10.13039/501100011033/FEDER, UE. This work was also partly funded by the Spanish Ministry of Science and Innovation/AEI/ 10.13039/501100011033/European Union NextGeneration/PRTR (grant TED2021-131871B-I00 [DIGIFRUIT project]).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Uncertainty analysis of a LiDAR-based MTLS point cloud using a high-resolution ground-truth

  • Lavaquiol Colell, Bernat
  • Llorens Calveras, Jordi
  • Sanz Cortiella, Ricardo
  • Arnó Satorra, Jaume
  • Escolà i Agustí, Alexandre
Poster Proceedings of the 14th European Conference on Precision Agriculture, 2-6 July 2023, Bologna, Italy, The study of plant geometry is crucial to design specific management by providing the optimal quantities of nutrients, fertilizers, pesticides and irrigation rates. Before the advent of the first 3D characterization systems, it was very laborious to obtain accurate commercial scale 3D crop data. Nowadays, there are sensing systems which allow 3D canopy characterization to be performed in a relatively simple and fast way. LiDAR (light detection and ranging) sensors have been widely used in agriculture. When 3D scanning techniques are used, it is essential to be aware of the total measurement error. One of the limitations when using real data is the absence of ground-truth (GT) to compare the obtained measurements . In a previous research [1], validated a high-resolution 3D point cloud on an actual defoliated tree obtained from RGB images and stereo-photogrammetry techniques. This accurate 3D point cloud can be used as digital ground-truth (DGT) to validate 3D LiDAR point. The accuracy of the scanning system includes the errors committed by the sensor, the positioning system (GNSS), the data acquisition set up, the point cloud generation algorithms and the georeferentiation of the DGT., The present study is part of the PAgPROTECT project PID2021-126648OB-I00 funded by the Spanish Ministry of Science and Innovation / AEI /10.13039/501100011033 / FEDER, UE.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Assessing automatic data processing algorithms for RGB-D cameras to predict fruit size and weight in apples

  • Miranda, Juan Carlos
  • Arnó Satorra, Jaume
  • Gené Mola, Jordi
  • Lordan Sanahuja, Jaume
  • Asin Jones, Luis
  • Gregorio López, Eduard
Data acquired using an RGB-D Azure Kinect DK camera were used to assess different automatic algorithms to estimate the size, and predict the weight of non-occluded and occluded apples. The programming of the algorithms included: (i) the extraction of images of regions of interest (ROI) using manual delimitation of bounding boxes or binary masks; (ii) estimating the lengths of the major and minor geometric axes for the purpose of apple sizing; and (iii) predicting the final weight by allometric modelling. In addition to the use of bounding boxes, the algorithms also allowed other post-mask settings (circles, ellipses and rotated rectangles) to be implemented, and different depth options (distance between the RGB-D camera and the fruits detected) for subsequent sizing through the application of the thin lens theory. Both linear and nonlinear allometric models demonstrated the ability to predict apple weight with a high degree of accuracy (R2 greater than 0.942 and RMSE < 16 g). With respect to non-occluded apples, the best weight predictions were achieved using a linear allometric model including both the major and minor axes of the apples as predictors. The mean absolute percentage error (MAPE) ranged from 5.1% to 5.7% with respective RMSE of 11.09 g and 13.02 g, depending to whether circles, ellipses, or bounding boxes were used to adjust fruit shape. The results were therefore promising and open up the possibility of implementing reliable in-field apple measurements in real time. Importantly, final weight prediction error and intermediate size estimation errors (from sizing algorithms) interact but in a way that is not easily quantifiable when weight allometric models with implicit prediction error are used. In addition, allometric models should be reviewed when applied to other apple cultivars, fruit development stages or even for different fruit growth conditions depending on canopy management., This work was partly funded by the Department of Research and Universities of the Generalitat de Catalunya (grants 2017, SGR 646 and 2021 LLAV 00088), by the Spanish Ministry of Science and Innovation / AEI/10.13039/501100011033 / ERDF (grants RTI2018-094222-B-I00 [PAgFRUIT project], PID2021-126648OB-I00 [PAgPROTECT project]) and by the Spanish Ministry of Science and Innovation / AEI/10.13039/ 501100011033 / European Union NextGeneration / PRTR (grantTED2021-131871B-I00 [DIGIFRUIT project]). We would also like to thank the Secretariat of Universities and Research of the Department of Business and Knowledge of the Generalitat de Catalunya and the European Social Fund (ESF) for financing Juan Carlos Miranda’s pre-doctoral fellowship (2020 FI_B 00586). The work of Jordi Gen´e-Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

LiDAR-derived indices and their relationship with productivity and oil quality attributes in high-density olive orchards

  • Sandonís Pozo, Leire
  • Rufat i Lamarca, Josep
  • Pascual Roca, Miquel
  • Villar Mir, Josep Ma.
  • Arnó Satorra, Jaume
  • Escolà i Agustí, Alexandre
  • Rosell Polo, Joan Ramon
  • Martínez Casasnovas, José Antonio
Optimising the leaf area index (LAI) and light exposure are key aspects to improve yield and quality in highdensity olive orchards. Traditionally, these characteristics are estimated using ceptometers, but this method is both labour intensive and dependent on light conditions, with limitations in effectiveness for precision horticulture management purposes. This study investigates the potential of four LiDAR-derived indices to: i) estimate the LAI and canopy light interception under different fertigation treatments, and ii) to assess their impact on yield and oil quality throughout a growing season in a high-density olive orchard. The treatments included two nitrogen dose rates (N0: 0 kg ha-¹ and N50: 50 kg ha-¹) combined with full irrigation (FI) and deficit irrigation (DI). The results provide strong evidence for the effectiveness of these indices in monitoring canopy response to treatments over time. The N50 and FI treatments increased the LAI and yield but led to poorer light attenuation within the canopy and lower oil quality. In contrast, N0 and DI promoted more light-permeable canopies and improved oil quality, with higher phenolic content, better oxidative stability and improved organoleptic oil attributes. These indices can improve the understanding of canopy light environments, help to optimise management practices, and ultimately improve the sustainability and profitability of intensive olive cultivation., This work was carried out with data acquired in the project IRTA2012–00059-C02–01 funded by Spain’s National Institute for Agricultural and Food Research and Technology, as well as in the projects AGL2010–22304-C04–03 and AGL2013–48297-C2–2-R funded by the Spanish Ministry of Economy and Competitiveness. The study was carried out under the framework of the grants RTI2018–094222-B-I00 (PAgFRUIT project) and PID2021–126648OB-I00 (PAgPROTECT project) through MCIN/AEI/10.13039/501100011033 and the European Union’s “ERDF, a way of making Europe”.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Simultaneous fruit detection and size estimation using multitask deep neural networks

  • Ferrer Ferrer, Mar
  • Ruiz Hidalgo, Javier
  • Gregorio López, Eduard
  • Vilaplana Besler, Verónica
  • Morros Rubió, Josep Ramon
  • Gené Mola, Jordi
The measurement of fruit size is of great interest to estimate the yield and predict the harvest resources in advance. This work proposes a novel technique for in-field apple detection and measurement based on Deep Neural Networks. The proposed framework was trained with RGB-D data and consists of an end-to-end multitask Deep Neural Network architecture specifically designed to perform the following tasks: 1) detection and segmentation of each fruit from its surroundings; 2) estimation of the diameter of each detected fruit. The methodology was tested with a total of 15,335 annotated apples at different growth stages, with diameters varying from 27 mm to 95 mm. Fruit detection results reported an F1-score for apple detection of 0.88 and a mean absolute error of diameter estimation of 5.64 mm. These are state-of-the-art results with the additional advantages of: a) using an end-to-end multitask trainable network; b) an efficient and fast inference speed; and c) being based on RGB-D data which can be acquired with affordable depth cameras. On the contrary, the main disadvantage is the need of annotating a large amount of data with fruit masks and diameter ground truth to train the model. Finally, a fruit visibility analysis showed an improvement in the prediction when limiting the measurement to apples above 65% of visibility (mean absolute error of 5.09 mm). This suggests that future works should develop a method for automatically identifying the most visible apples and discard the prediction of highly occluded fruits., This work was partly funded by the Departament de Recerca i Universitats de la Generalitat de Catalunya (grant 2021 LLAV 00088), the Spanish Ministry of Science, Innovation and Universities (grants RTI2018-094222-B-I00[PAgFRUIT project], PID2021-126648OB-I00 [PAgPROTECT project] and PID2020- 117142 GB-I00 [DeeLight project] by MCIN/AEI/10.13039/ 501100011033 and by “ERDF, a way of making Europe”, by the European Union). The work of Jordi Gene Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

A new Leafiness-LiDAR index to estimate light interception in intensive olive orchards

  • Sandonís Pozo, Leire
  • Martínez Casasnovas, José Antonio
  • Escolà i Agustí, Alexandre
  • Rosell Polo, Joan Ramon
  • Rufat i Lamarca, Josep
  • Pascual Roca, Miquel
The 14th European Conference on Precision Agriculture, July 2-6, 2023, Bologna, Italy, Canopy light interception constitutes an important yield limiting factor in high density olive orchards. However, its characterisation still implies laborious measurements. A new index, the Leafiness-LiDAR index (LLI), is presented as a LAI estimator. LLI combines LiDAR-derived parameters: Cross-Section and Leafiness from 3D point clouds. To validate the results, photosynthetically active radiation (PAR) measurements, canopy volume, yield and quality parameters were collected and analysed. LLI showed significant correlations both with PAR and canopy volume (r = 0.8) and quality parameters (r = -0.6). LLI may be useful as an early decision canopy monitoring tool in the framework of Precision Fructiculture., This work was carried out with data acquired in the project IRTA2012-00059-C02-01 funded by the Instituto Nacional de Investigación y Tecnología Agraria y Alimentaria, as well as in the projects AGL2010-22304-C04-03 and AGL2013-48297-C2-2-R funded by the Spanish Ministry of Economy and Competitiveness and developed under the frame of the grants RTI2018-094222-B-I00 (PAgFRUIT project) and PID2021-126648OB-I00 (PAgPROTECT project) by MCIN/AEI/10.13039/501100011033 and by “ERDF, a way of making Europe”, by the European Union.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Video-based fruit detection and tracking: effects of scanning conditions on fruit load estimation

  • Felip-Pomés, Marc
  • Gené-Mola, Jordi
  • Arnó, Jaume
  • Lordan, Jaume
  • Ruiz-Hidalgo, Javier
  • Morros, Josep-Ramon
  • Arasanz-Malo, Miguel
  • Net-Barnés, Francesc
  • Gregorio, Eduard
Automated fruit counting and yield estimation systems are necessary for efficient orchard management. This study presents a computer vision system based on video multi-object-tracking for fruit load estimation in apple orchards and provides a comprehensive analysis of the system performance under diverse scanning conditions. The system integrates fruit detection, tracking, localization within orchard, and fruit load map generation. Experiments were carried out in an experimental apple orchard containing 420 apple trees. Data was collected with two different RGB-D sensors (Azure Kinect DK and ZED 2) at three different scanning distances (125 cm, 175 cm, and 225 cm) on two different dates prior to the harvest. Comparing the two evaluated sensors, Azure Kinect provided more consistent performance across different dates. Results also show that the longer scanning distance improves accuracy due to seeing the full tree view gives better fruit counts than close partial views. Between the two dates, best results were achieved near harvest due to fruit color at this stage, achieving a Mean Absolute Percentage Error (MAPE) of 6.91 % and a determination coefficient (R2) of 0.733 (using ZED2 sensor at 225 cm distance). Finally, a test comparing scanning from one or both sides of the tree row showed that bilateral scanning improved fruit load estimation at the stretch level by incorporating information from both sides of the canopy. The results of this work demonstrate the effectiveness of the video fruit tracking systems as a useful tool for automating fruit load estimation., This work was partly funded by the Department of Research and Universities of the Generalitat de Catalunya (grants 2017, SGR 646 and 2021 LLAV 00088), by the Spanish Ministry of Science and Innovation /AEI/10.13039/501100011033 / ERDF (grants RTI2018-094222-B-I00 [PAgFRUIT project], PID2021-126648OB-I00 [PAgPROTECT project], PID2024-161868OB-I00 [C3DRUM project], and PID2024-156495OR-C31 [YIELD4CAST]), by the Spanish Ministry of Science and Innovation / AEI/10.13039/501100011033 / European Union NextGeneration / PRTR (grant TED2021-131871B-I00 [DIGIFRUIT project]), and by the Agrotecnio-IRTA Joint Projects Call 2025 (PhotonFruit project). The work of Jordi Gené-Mola was partially supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU., info:eu-repo/semantics/publishedVersion
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Looking behind occlusions: A study on amodal segmentation for robust on-tree apple fruit size estimation

  • Gené Mola, Jordi
  • Ferrer Ferrer, Mar
  • Gregorio López, Eduard
  • Blok, Pieter
  • Hemming, Jochen
  • Morros Rubió, Josep Ramon
  • Rosell Polo, Joan Ramon
  • Vilaplana Besler, Verónica
  • Ruiz Hidalgo, Javier
The detection and sizing of fruits with computer vision methods is of interest because it provides relevant information to improve the management of orchard farming. However, the presence of partially occluded fruits limits the performance of existing methods, making reliable fruit sizing a challenging task. While previous fruit segmentation works limit segmentation to the visible region of fruits (known as modal segmentation), in this work we propose an amodal segmentation algorithm to predict the complete shape, which includes its visible and occluded regions. To do so, an end-to-end convolutional neural network (CNN) for sim ultaneous modal and amodal instance segmentation was implemented. The predicted amodal masks were used to estimate the fruit diameters in pixels. Modal masks were used to identify the visible region and measure the distance between the apples and the camera using the depth image. Finally, the fruit diameters in millimetres (mm) were computed by applying the pinhole camera model. The method was developed with a Fuji apple dataset consisting of 3925 RGB-D images acquired at different growth stages with a total of 15,335 annotated apples, and was subsequently tested in a case study to measure the diameter of Elstar apples at different growth stages. Fruit detection results showed an F1-score of 0.86 and the fruit diameter results reported a mean absolute error (MAE) of 4.5 mm and R2 = 0.80 irrespective of fruit visibility. Besides the diameter estimation, modal and amodal masks were used to automatically determine the percentage of visibility of measured apples. This feature was used as a confidence value, improving the diameter estimation to MAE = 2.93 mm and R2 = 0.91 when limiting the size estimation to fruits detected with a visibility higher than 60%. The main advantages of the present methodology are its robustness for measuring partially occluded fruits and the capability to determine the visibility percentage. The main limitation is that depth images were generated by means of photogrammetry methods, which limits the efficiency of data acquisition. To overcome this limitation, future works should consider the use of commercial RGB-D sensors. The code and the dataset used to evaluate the method have been made publicly available at https://github.com/GRAP-UdL-AT/Amodal_Fruit_Sizing., This work was partly funded by the Departament de Recerca i Universitats de la Generalitat de Catalunya (grant 2021 LLAV 00088), the Spanish Ministry of Science, Innovation and Universities (grants RTI2018-094222-B-I00 [PAgFRUIT project], PID2021-126648OB-I00 [PAgPROTECT project] and PID2020-117142GB-I00 [DeeLight project] by MCIN/AEI/10.13039/501100011033 and by “ERDF, a way of making Europe”, by the European Union). The work of Jordi Gen´e Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU. We would also like to thank Nufri (especially Santiago Salamero and Oriol Morreres) for their support during data acquisition, and Pieter van Dalfsen and Dirk de Hoog from Wageningen University & Research for additional data collection used in the case study.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Análisis de los efectos de la sequía en manzanos a través del índice Leafiness-LiDAR

  • Sandonís Pozo, Leire
  • Martínez Casasnovas, José Antonio
  • Pascual Roca, Miquel
La sequía es un factor ambiental significativo que puede afectar de manera considerable el desarrollo y la productividad los cultivos frutícolas. En manzanos (Malus sylvestris L.Mill.), diversos autores han demostrado que la productividad y la calidad de los frutos está directamente relacionada con la cantidad de radiación solar absorbida por las copas. La gestión adecuada del dosel foliar durante períodos de sequía es esencial para minimizar sus consecuencias sobre la productividad de los manzanos. En este sentido, los sensores LiDAR (Light Detection And Ranging), utilizados en Fruticultura de Precisión para la caracterización 3D de la vegetación, pueden aportar la información necesaria para cuantificar las características geométricas y estructurales del dosel foliar de las plantaciones. Concretamente, en el presente trabajo se ha utilizado el índice Leafiness-LiDAR Index (LLI) medido en una plantación comercial intensiva de manzanos (Malus sylvestris L.Mill.) cv UEB 3264/2 Opal®, en la que se diseñó un experimento con diferentes dosis regadío a fin de simular las condiciones de sequía (desde condiciones deficitarias en diferentes períodos hasta riego completo). Los resultados obtenidos del análisis LiDAR permiten estimar adecuadamente el índice de área foliar (LAI) de los árboles, pudiendo relacionar estos indicadores con diversos e importantes atributos cuantitativos: crecimiento vegetativo, frondosidad del seto y rendimiento., Este trabajo es fruto del proyecto PID2021-126648OB-I00 (PAgPROTECT) concedido por MCIN/AEI/10.13039/501100011033 y por “ERDF, a way of make Europe”, de la Unión Europea.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

AKFruitYield: Modular benchmarking and video analysis software for Azure Kinect cameras for fruit size and fruit yield estimation in apple orchards

  • Miranda, Juan Carlos
  • Arnó Satorra, Jaume
  • Gené Mola, Jordi
  • Fountas, Spyros
  • Gregorio López, Eduard
AKFruitYield is a modular software that allows orchard data from RGB-D Azure Kinect cameras to be processed for fruit size and fruit yield estimation. Specifically, two modules have been developed: i) AK_SW_BENCHMARKER that makes it possible to apply different sizing algorithms and allometric yield prediction models to manually labelled color and depth tree images; and ii) AK_VIDEO_ANALYSER that analyses videos on which to automatically detect apples, estimate their size and predict yield at the plot or per hectare scale using the appropriate algorithms. Both modules have easy-to-use graphical interfaces and provide reports that can subsequently be used by other analysis tools., This work was partly funded by the Department of Research and Universities of the Generalitat de Catalunya (grants 2017 SGR 646) and by the Spanish Ministry of Science and Innovation/AEI/10.13039/501100011033/ERDF (grant RTI2018–094222-B-I00 [PAgFRUIT project] and PID2021–126648OB-I00 [PAgPROTECT project]). The Secretariat of Universities and Research of the Department of Business and Knowledge of the Generalitat de Catalunya and European Social Fund (ESF) are also thanked for financing Juan Carlos Miranda's pre-doctoral fellowship (2020 FI_B 00586). The work of Jordi Gené-Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU. The authors would also like to thank the Institut de Recerca i Tecnologia Agroalimentàries (IRTA) for allowing the use of their experimental fields, and in particular Dr. Luís Asín and Dr. Jaume Lordán who have contributed to the success of this work.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Video-based fruit detection and tracking: effects of scanning conditions on fruit load estimation

  • Felip-Pomés, , Marc
  • Gené Mola, Jordi
  • Arnó Satorra, Jaume
  • Lordan Sanahuja, Jaume
  • Ruiz Hidalgo, Javier
  • Morros Rubió, Josep Ramon
  • Arasanz-Malo,Miguel
  • Net Barnés, Francesc
  • Gregorio López, Eduard
Automated fruit counting and yield estimation systems are necessary for efficient orchard management. This study presents a computer vision system based on video multi-object-tracking for fruit load estimation in apple orchards and provides a comprehensive analysis of the system performance under diverse scanning conditions. The system integrates fruit detection, tracking, localization within orchard, and fruit load map generation. Experiments were carried out in an experimental apple orchard containing 420 apple trees. Data was collected with two different RGB-D sensors (Azure Kinect DK and ZED 2) at three different scanning distances (125 cm, 175 cm, and 225 cm) on two different dates prior to the harvest. Comparing the two evaluated sensors, Azure Kinect provided more consistent performance across different dates. Results also show that the longer scanning distance improves accuracy due to seeing the full tree view gives better fruit counts than close partial views. Between the two dates, best results were achieved near harvest due to fruit color at this stage, achieving a Mean Absolute Percentage Error (MAPE) of 6.91 % and a determination coefficient (R2) of 0.733 (using ZED2 sensor at 225 cm distance). Finally, a test comparing scanning from one or both sides of the tree row showed that bilateral scanning improved fruit load estimation at the stretch level by incorporating information from both sides of the canopy. The results of this work demonstrate the effectiveness of the video fruit tracking systems as a useful tool for automating fruit load estimation., This work was partly funded by the Department of Research and Universities of the Generalitat de Catalunya (grants 2017, SGR 646 and 2021 LLAV 00088), by the Spanish Ministry of Science and Innovation /AEI/10.13039/501100011033 / ERDF (grants RTI2018-094222-B-I00 [PAgFRUIT project], PID2021-126648OB-I00 [PAgPROTECT project], PID2024-161868OB-I00 [C3DRUM project], and PID2024-156495OR-C31 [YIELD4CAST]), by the Spanish Ministry of Science and Innovation / AEI/10.13039/501100011033 / European Union NextGeneration / PRTR (grant TED2021-131871B-I00 [DIGIFRUIT project]), and by the Agrotecnio-IRTA Joint Projects Call 2025 (PhotonFruit project). The work of Jordi Gené-Mola was partially supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Fruit sizing using AI: A review of methods and challenges

  • Miranda, Juan Carlos
  • Gené Mola, Jordi
  • Zude-Sasse, Manuela
  • Tsoulias, Nikos
  • Escolà i Agustí, Alexandre
  • Arnó Satorra, Jaume
  • Rosell Polo, Joan Ramon
  • Sanz Cortiella, Ricardo
  • Martínez Casasnovas, José Antonio
  • Gregorio López, Eduard
Fruit size at harvest is an economically important variable for high-quality table fruit production in orchards and vineyards. In addition, knowing the number and size of the fruit on the tree is essential in the framework of precise production, harvest, and postharvest management. A prerequisite for analysis of fruit in a real-world environment is the detection and segmentation from background signal. In the last five years, deep learning convolutional neural network have become the standard method for automatic fruit detection, achieving F1-scores higher than 90 %, as well as real-time processing speeds. At the same time, different methods have been developed for, mainly, fruit size and, more rarely, fruit maturity estimation from 2D images and 3D point clouds. These sizing methods are focused on a few species like grape, apple, citrus, and mango, resulting in mean absolute error values of less than 4 mm in apple fruit. This review provides an overview of the most recent methodologies developed for in-field fruit detection/counting and sizing as well as few upcoming examples of maturity estimation. Challenges, such as sensor fusion, highly varying lighting conditions, occlusions in the canopy, shortage of public fruit datasets, and opportunities for research transfer, are discussed., his work was partly funded by the Department of Research and Universities of the Generalitat de Catalunya (grants 2017 SGR 646 and 2021 LLAV 00088) and by the Spanish Ministry of Science and Innovation / AEI/10.13039/501100011033 / FEDER (grants RTI2018-094222-B-I00 [PAgFRUIT project] and PID2021-126648OB-I00 [PAgPROTECT project]). The Secretariat of Universities and Research of the Department of Business and Knowledge of the Generalitat de Catalunya and European Social Fund (ESF) are also thanked for financing Juan Carlos Miranda’s pre-doctoral fellowship (2020 FI_B 00586). The work of Jordi Gené-Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Video-Based Fruit Detection and Tracking for Apple Counting and Mapping

  • Gené Mola, Jordi
  • Felip Pomés, Marc
  • Net Barnés, Francesc
  • Morros Rubió, Josep Ramon
  • Miranda, Juan Carlos
  • Arnó Satorra, Jaume
  • Asin Jones, Luis
  • Lordan Sanahuja, Jaume
  • Ruiz Hidalgo, Javier
  • Gregorio López, Eduard
Automatic fruit counting systems have garnered interest from farmers and agronomists to monitor fruit production, predict yields in advance, and identify production variability across orchards. However, accurately counting fruits poses challenges, particularly due to occlusions. This study proposes a multi-view sensing approach using continuous motion videos captured by a camera moved along the row of trees, followed by fruit detection in all video frames and application of Multi-Object Tracking (MOT) algorithms to prevent double-counting. Three tracking methods, namely SORT, DeepSORT, and ByteTrack, are compared for fruit counting using the YOLOv5x object detector. The methodology is applied to map fruit production in an experimental apple orchard at two different dates: four weeks and one week before harvest. The results demonstrate that ByteTrack (MOTA=0.682; IDF1=0.837; HOTA=0.689) outperforms SORT and DeepSORT, indicating its superior tracking performance. Computational efficiency analysis reveals similar processing times between SORT and ByteTrack (about 15 ms), while DeepSORT requires significantly more processing time per image (128 ms). Fruit counting evaluation shows reasonably accurate yield predictions on both dates, with reduced errors and improved performance closer to the harvest date (MAPE=7.47 %; R2=0.70). The system proves effective in estimating orchard fruit production using computer vision technology, offering valuable insights for yield forecasting. These findings contribute to optimizing fruit production and supporting precision agriculture practices. The code and the dataset have been made publicly available and a video visualization of results is accessible at http://www.grap.udl.cat/en/publications/video_fruit_counting., This work was partly funded by the Departament de Recerca i Universitats de la Generalitat de Catalunya (grant 2021 LLAV 00088), the Spanish Ministry of Science, Innovation and Universities (grants RTI2018-094222-B-I00 [PAgFRUIT project], PID2021-126648OB-I00 [PAgPROTECT project] and PID2020-117142GB-I00 [DeeLight project] by MCIN/AEI/10.13039/501100011033 and by “ERDF, a way of making Europe”, by the European Union). The work of Jordi Gené Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU. The Secretariat of Universities and Research of the Department of Business and Knowledge of the Generalitat de Catalunya and European Social Fund (ESF) are also thanked for financing Juan Carlos Miranda’s pre doctoral fellowship (2020 FI_B 00586).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Parameter configuration to maximize accuracy in point clouds acquired with lidar-based terrestrial mobile laser scanners in fruit tree orchards

  • Lavaquiol Colell, Bernat
  • Escolà i Agustí, Alexandre
  • Arnó Satorra, Jaume
  • Grau, Joel
  • Ninot, Jérôme
  • Gómez, David
  • Llorens Calveras, Jordi
The aim of this paper is to determine the optimal parameter configuration to minimize LiDAR point cloud errors and maximize accuracy for agronomic applications in field conditions in the framework of Precision Agriculture. The structure and geometry of trees are closely linked to vegetative parameters and productivity in fruit orchards. Therefore, achieving precise and accurate geometric characterization of tree canopies is fundamental for developing site-specific management strategies for resource optimization while reducing costs and environmental impacts associated with agricultural practices. Among the various sensing technologies available, LiDAR-based sensors have emerged as one of the preferred options for obtaining accurate geometric data in orchards. For this reason, the paper focuses on analyzing how different parameter configurations used during point cloud production contribute to reconstruction and location errors. Specifically, it examines the influence of type of SLAM algorithm (triple resolution or SLAM3 and feature or SLAMF), the GNSS constellations in use, orbit and clock products, trajectory correction (PPK or TC) and navigation constraint resolution, and the distance to the GNSS base station. Regarding the reconstruction error, since the maximum difference between the best configuration (featuring SLAM3 and PPK) and the worst (featuring SLAMF and TC) is just 0.006 m, all parameter configurations are capable of efficiently reconstructing point clouds. Location error analysis showed that GPS-based solutions consistently exceeded 0.1 m, while GNSS-based solutions - involving any combination of two or more constellations among GPS, GLONASS, Galileo, and BeiDou - remained below 0.07 m. The best configurations include SLAM3 paired with PPK, regardless of the navigation constraint resolution, achieving an error of 0.032 m in the most favorable case. Although PPK trajectories performed better in GNSS-based solutions, they were less effective than TC when only GPS-based solutions were available. Additionally, using PPK trajectories ensures better time efficiency, as they are a prerequisite for generating TC trajectories. SLAM3 outperformed SLAMF under the tested conditions. This study represents a significant advancement in the optimization of LiDAR-derived point clouds generation. By identifying and fine-tuning key parameter configurations, it effectively minimizes both reconstruction and location errors, while simultaneously enhancing the temporal efficiency of the point cloud production process., The present study is part of the Project PAgPROTECT (PID2021–126648OB-I00) funded by MICIU/AEI /10.13039/501100011033 and by FEDER/ERDF, EU.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Leafiness-LiDAR index and NDVI for identification of temporal patterns in super-intensive almond orchards as response to different management strategies

  • Sandonís Pozo, Leire
  • Oger, Baptiste
  • Tisseyre, Bruno
  • Llorens Calveras, Jordi
  • Escolà i Agustí, Alexandre
  • Pascual Roca, Miquel
  • Martínez Casasnovas, José Antonio
The use of super-intensive orchards is a growing trend in fruit production. The present study aims to improve management of these cropping systems by focusing on how agronomic decisions impact orchard dynamics in the short to medium term and by providing a decision-support approach based on stable temporal patterns from previous seasons. A multitemporal study using remote sensing and LiDAR was conducted in a commercial almond orchard over four growing seasons (2019-2022) to determine the optimal timing of image acquisition for variable pre-harvest treatments. A model-based clustering (mclust) was applied to optimal Sentinel-2 NDVI maps and apparent soil electrical conductivity (ECa) data, interpolated to the pixel centroids of Sentinel-2 image grids, to delineate potential management zones (PMZs). The leafiness-LiDAR index (LLI), a leaf area index (LAI) estimator, was obtained as ground truth after summer pruning and before harvesting, showing a significant influence of fertigation and pruning on the LAI, with summer pruning particularly influencing orchard dynamics. The optimal time for NDVI mapping was found to be two months after summer pruning in productive years and two weeks after in unproductive years. The delineated PMZs were consistent across seasons and corresponded to significant LAI differences. This method could contribute to improving resource management and sustainability in super-intensive commercial orchards., This research was funded by Grant RTI2018–094222-B-I00 (PAgFRUIT project) and PID2021–126648OB-I00 (PAgPROTECT project) by MCIN/AEI/10.13039/501100011033 and by “ERDF, a way of making Europe” of the European Union. The authors wish to thank Alrasa Agraria S.L. (Raimat, Lleida), the Centre for Technological and Industrial Development (Ministry of Science and Innovation, Government of Spain) and EuroChem Iberia S.A. for providing funds and experimental facilities, and to the members of the Research Group in AgroICT and Precision Agriculture (GRAP) of the Universitat de Lleida for their collaboration in data acquisition.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

AmodalAppleSize_RGB-D dataset: RGB-D images of apple trees annotated with modal and amodal segmentation masks for fruit detection, visibility and size estimation

  • Gené Mola, Jordi
  • Ferrer Ferrer, Mar
  • Hemming, Jochen
  • Dalfsen, Pieter van
  • Hoog, Dirk de
  • Sanz Cortiella, Ricardo
  • Rosell Polo, Joan Ramon
  • Vilaplana Besler, Verónica
  • Ruiz Hidalgo, Javier
  • Gregorio López, Eduard
The present dataset comprises a collection of RGB-D apple tree images that can be used to train and test computer vision-based fruit detection and sizing methods. This dataset encompasses two distinct sets of data obtained from a Fuji and an Elstar apple orchards. The Fuji apple orchard sub-set consists of 3925 RGB-D images containing a total of 15,335 apples annotated with both modal and amodal apple segmentation masks. Modal masks denote the visible portions of the apples, whereas amodal masks encompass both visible and occluded apple regions. Notably, this dataset is the first public resource to incorporate on-tree fruit amodal masks. This pioneering inclusion addresses a critical gap in existing datasets, enabling the development of robust automatic fruit sizing methods and accurate fruit visibility estimation, particularly in the presence of partial occlusions. Besides the fruit segmentation masks, the dataset also includes the fruit size (calliper) ground truth for each annotated apple. The second sub-set comprises 2731 RGB-D images capturing five Elstar apple trees at four distinct growth stages. This sub-set includes mean diameter information for each tree at every growth stage and serves as a valuable resource for evaluating fruit sizing methods trained with the first sub-set. The present data was employed in the research paper titled “Looking behind occlusions: a study on amodal segmentation for robust on-tree apple fruit size estimation”., This work was partly funded by the Departament de Recerca i Universitats de la Generalitat de Catalunya (grant 2021 LLAV 00088 ), the Spanish Ministry of Science, Innovation and Universities (grants RTI2018-094222-B-I00 [PAgFRUIT project], PID2021-126648OB-I00 [PAgPROTECT project] and PID2020-117142GB-I00 [DeeLight project] by MCIN/AEI/10.13039/501100011033 and by “ERDF, a way of making Europe”, by the European Union). Data presented in this paper is also part of a Public Private Partnership project Precisie Tuinbouw, WP Fruit 4.0 (PPS KV 1604-025) and financed by Topsector Tuinbouw & Uitgangsmateriaal and various private companies. The work of Jordi Gené Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU. We would also like to thank Nufri (especially Santiago Salamero and Oriol Morreres) for their support during data acquisition.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Low-cost terrestrial photogrammetry for orchard sidewards 3D reconstruction

  • Martínez Casasnovas, José Antonio
  • Rosell Tarragó, Miquel
  • Rosell Polo, Joan Ramon
  • Sanz Cortiella, Ricardo
  • Gregorio López, Eduard
  • Gené Mola, Jordi
  • Arnó Satorra, Jaume
  • Plata Moreno, José Manuel
  • Escolà i Agustí, Alexandre
The present study was part of the grants RTI2018-094222-B-I00 and PID2021-126648OB-I00 (PAgFRUIT and PAgPROTECT projects), funded by MCIN/AEI/10.13039/501100011033 and by “ERDF, a way of making Europe”, by the European Union
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Mobile terrestrial laser scanner vs. UAV photogrammetry to estimate woody crop canopy parameters - Part 2: Comparison for different crops and training systems

  • Torres-Sánchez, Jorge
  • Escolà i Agustí, Alexandre
  • de Castro, Ana I.
  • López-Granados, Francisca
  • Rosell Polo, Joan Ramon
  • Sebé Feixas, Francesc
  • Jiménez-Brenes, Francisco M.
  • Sanz Cortiella, Ricardo
  • Gregorio López, Eduard
  • Peña, José M.
The measurement of geometric canopy parameters in woody crops is an important task in Precision Agriculture because of their correlation with crop condition and productivity. In recent years, several technological approaches have been developed as an alternative to manual measurements, which are time- and labour-consuming. Two of the most commonly used 3D canopy characterization technologies are mobile terrestrial laser scanning (MTLS) based on light detection and ranging (LiDAR) sensors, and digital aerial photogrammetry (DAP) using imagery from uncrewed aerial vehicles (UAVs). Although both are state-of-the-art and have been fully tested and validated, a complete comparison between their geometric canopy parameter estimations in different woody crops and training systems has not been carried out. For this reason, a set of geometric parameters (canopy height, projected area, and volume) of a vineyard, an intensive peach orchard, and an intensive pear orchard were measured using UAV-DAP and MTLS-LiDAR. A comparison between both kinds of measurements was performed, accounting for the length of the sections in which the crop hedgerows were divided to extract the geometric parameters. Measurements from the UAV and the MTLS were highly correlated (R2 from 0.82 to 0.94) when considering the data from the three crops together, and the correlations were higher when analysing longer row sections. The canopy geometric parameters estimated using the MTLS-LiDAR always had higher values than those from the UAV-DAP. The results presented in this work provide useful data for a more informed selection of technological approaches for 3D crop characterization in Precision Fruticulture and high-throughput phenotyping., This research was funded by the projects DECIMAL (PID2020-113229RB-C41), PRODIGIA (PID2020-113229RB-C44), AGVANCE (AGL2013-48297-C2-2-R), and PAgPROTECT (PID2021-126648OB-I00), from the Spanish Ministry of Science and Innovation (MCIN/AEI/10.13039/501100011033). The authors also wish to thank Codorníu SA, celler Raimat and IRTA for having allowed the use of their vineyards, pear and peach orchards to conduct the trials described in this work.