Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 19
Encontrada(s) 1 página(s)

IRTA Pubpro. Open Digital Archive
Publicación Artículo científico (JournalArticle)

Assessing automatic data processing algorithms for RGB-D cameras to predict fruit size and weight in apples

  • Miranda, Juan C.
  • Arnó, Jaume
  • Gené-Mola, Jordi
  • Lordan, Jaume
  • Asín, Luis
  • Gregorio, Eduard
Data acquired using an RGB-D Azure Kinect DK camera were used to assess different automatic algorithms to estimate the size, and predict the weight of non-occluded and occluded apples. The programming of the algorithms included: (i) the extraction of images of regions of interest (ROI) using manual delimitation of bounding boxes or binary masks; (ii) estimating the lengths of the major and minor geometric axes for the purpose of apple sizing; and (iii) predicting the final weight by allometric modelling. In addition to the use of bounding boxes, the algorithms also allowed other post-mask settings (circles, ellipses and rotated rectangles) to be implemented, and different depth options (distance between the RGB-D camera and the fruits detected) for subsequent sizing through the application of the thin lens theory. Both linear and nonlinear allometric models demonstrated the ability to predict apple weight with a high degree of accuracy (R2 greater than 0.942 and RMSE < 16 g). With respect to non-occluded apples, the best weight predictions were achieved using a linear allometric model including both the major and minor axes of the apples as predictors. The mean absolute percentage error (MAPE) ranged from 5.1% to 5.7% with respective RMSE of 11.09 g and 13.02 g, depending to whether circles, ellipses, or bounding boxes were used to adjust fruit shape. The results were therefore promising and open up the possibility of implementing reliable in-field apple measurements in real time. Importantly, final weight prediction error and intermediate size estimation errors (from sizing algorithms) interact but in a way that is not easily quantifiable when weight allometric models with implicit prediction error are used. In addition, allometric models should be reviewed when applied to other apple cultivars, fruit development stages or even for different fruit growth conditions depending on canopy management., This work was partly funded by the Department of Research and Universities of the Generalitat de Catalunya (grants 2017, SGR 646 and 2021 LLAV 00088), by the Spanish Ministry of Science and Innovation / AEI/10.13039/501100011033 / ERDF (grants RTI2018-094222-B-I00 [PAgFRUIT project], PID2021-126648OB-I00 [PAgPROTECT project]) and by the Spanish Ministry of Science and Innovation / AEI/10.13039/501100011033 / European Union NextGeneration / PRTR (grantTED2021-131871B-I00 [DIGIFRUIT project]). We would also like to thank the Secretariat of Universities and Research of the Department of Business and Knowledge of the Generalitat de Catalunya and the European Social Fund (ESF) for financing Juan Carlos Miranda’s pre-doctoral fellowship (2020 FI_B 00586). The work of Jordi Gené-Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU., info:eu-repo/semantics/publishedVersion
IRTA Pubpro. Open Digital Archive
Publicación Artículo científico (JournalArticle)

Fruit sizing using AI: A review of methods and challenges

  • Miranda, Juan C.
  • Gené-Mola, Jordi
  • Zude-Sasse, Manuela
  • Tsoulias, Nikos
  • Escolà, Alexandre
  • Arnó, Jaume
  • Rosell-Polo, Joan R.
  • Sanz-Cortiella, Ricardo
  • Martínez-Casasnovas, José A.
  • Gregorio, Eduard
Fruit size at harvest is an economically important variable for high-quality table fruit production in orchards and vineyards. In addition, knowing the number and size of the fruit on the tree is essential in the framework of precise production, harvest, and postharvest management. A prerequisite for analysis of fruit in a real-world environment is the detection and segmentation from background signal. In the last five years, deep learning convolutional neural network have become the standard method for automatic fruit detection, achieving F1-scores higher than 90 %, as well as real-time processing speeds. At the same time, different methods have been developed for, mainly, fruit size and, more rarely, fruit maturity estimation from 2D images and 3D point clouds. These sizing methods are focused on a few species like grape, apple, citrus, and mango, resulting in mean absolute error values of less than 4 mm in apple fruit. This review provides an overview of the most recent methodologies developed for in-field fruit detection/counting and sizing as well as few upcoming examples of maturity estimation. Challenges, such as sensor fusion, highly varying lighting conditions, occlusions in the canopy, shortage of public fruit datasets, and opportunities for research transfer, are discussed., This work was partly funded by the Department of Research and Universities of the Generalitat de Catalunya (grants 2017 SGR 646 and 2021 LLAV 00088) and by the Spanish Ministry of Science and Innovation / AEI/10.13039/501100011033 / FEDER (grants RTI2018-094222-B-I00 [PAgFRUIT project] and PID2021-126648OB-I00 [PAgPROTECT project]). The Secretariat of Universities and Research of the Department of Business and Knowledge of the Generalitat de Catalunya and European Social Fund (ESF) are also thanked for financing Juan Carlos Miranda’s pre-doctoral fellowship (2020 FI_B 00586). The work of Jordi Gené-Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU., info:eu-repo/semantics/publishedVersion
IRTA Pubpro. Open Digital Archive
Publicación Artículo científico (JournalArticle)

LiDAR-derived indices and their relationship with productivity and oil quality attributes in high-density olive orchards

  • Sandonis-Pozo, L.
  • Rufat, Josep
  • Pascual, M.
  • Villar, J.M.
  • Arnó, J.
  • Escolà, A.
  • Rosell-Polo, J.R.
  • Martínez-Casasnovas, J.A.
Optimising the leaf area index (LAI) and light exposure are key aspects to improve yield and quality in high-density olive orchards. Traditionally, these characteristics are estimated using ceptometers, but this method is both labour intensive and dependent on light conditions, with limitations in effectiveness for precision horticulture management purposes. This study investigates the potential of four LiDAR-derived indices to: i) estimate the LAI and canopy light interception under different fertigation treatments, and ii) to assess their impact on yield and oil quality throughout a growing season in a high-density olive orchard. The treatments included two nitrogen dose rates (N0: 0 kg ha-¹ and N50: 50 kg ha-¹) combined with full irrigation (FI) and deficit irrigation (DI). The results provide strong evidence for the effectiveness of these indices in monitoring canopy response to treatments over time. The N50 and FI treatments increased the LAI and yield but led to poorer light attenuation within the canopy and lower oil quality. In contrast, N0 and DI promoted more light-permeable canopies and improved oil quality, with higher phenolic content, better oxidative stability and improved organoleptic oil attributes. These indices can improve the understanding of canopy light environments, help to optimise management practices, and ultimately improve the sustainability and profitability of intensive olive cultivation., This work was carried out with data acquired in the project IRTA2012–00059-C02–01 funded by Spain’s National Institute for Agricultural and Food Research and Technology, as well as in the projects AGL2010–22304-C04–03 and AGL2013–48297-C2–2-R funded by the Spanish Ministry of Economy and Competitiveness. The study was carried out under the framework of the grants RTI2018–094222-B-I00 (PAgFRUIT project) and PID2021–126648OB-I00 (PAgPROTECT project) through MCIN/AEI/10.13039/501100011033 and the European Union’s “ERDF, a way of making Europe”., info:eu-repo/semantics/publishedVersion
IRTA Pubpro. Open Digital Archive
Publicación Artículo científico (JournalArticle)

Looking behind occlusions: A study on amodal segmentation for robust on-tree apple fruit size estimation

  • Gené-Mola, Jordi
  • Ferrer-Ferrer, Mar
  • Gregorio, Eduard
  • Blok, Pieter M.
  • Hemming, Jochen
  • Morros, Josep-Ramon
  • Rosell-Polo, Joan R.
  • Vilaplana, Verónica
  • Ruiz-Hidalgo, Javier
The detection and sizing of fruits with computer vision methods is of interest because it provides relevant information to improve the management of orchard farming. However, the presence of partially occluded fruits limits the performance of existing methods, making reliable fruit sizing a challenging task. While previous fruit segmentation works limit segmentation to the visible region of fruits (known as modal segmentation), in this work we propose an amodal segmentation algorithm to predict the complete shape, which includes its visible and occluded regions. To do so, an end-to-end convolutional neural network (CNN) for simultaneous modal and amodal instance segmentation was implemented. The predicted amodal masks were used to estimate the fruit diameters in pixels. Modal masks were used to identify the visible region and measure the distance between the apples and the camera using the depth image. Finally, the fruit diameters in millimetres (mm) were computed by applying the pinhole camera model. The method was developed with a Fuji apple dataset consisting of 3925 RGB-D images acquired at different growth stages with a total of 15,335 annotated apples, and was subsequently tested in a case study to measure the diameter of Elstar apples at different growth stages. Fruit detection results showed an F1-score of 0.86 and the fruit diameter results reported a mean absolute error (MAE) of 4.5 mm and R2 = 0.80 irrespective of fruit visibility. Besides the diameter estimation, modal and amodal masks were used to automatically determine the percentage of visibility of measured apples. This feature was used as a confidence value, improving the diameter estimation to MAE = 2.93 mm and R2 = 0.91 when limiting the size estimation to fruits detected with a visibility higher than 60%. The main advantages of the present methodology are its robustness for measuring partially occluded fruits and the capability to determine the visibility percentage. The main limitation is that depth images were generated by means of photogrammetry methods, which limits the efficiency of data acquisition. To overcome this limitation, future works should consider the use of commercial RGB-D sensors. The code and the dataset used to evaluate the method have been made publicly available at https://github.com/GRAP-UdL-AT/Amodal_Fruit_Sizing, This work was partly funded by the Departament de Recerca i Universitats de la Generalitat de Catalunya (grant 2021 LLAV 00088), the Spanish Ministry of Science, Innovation and Universities (grants RTI2018-094222-B-I00 [PAgFRUIT project], PID2021-126648OB-I00 [PAgPROTECT project] and PID2020-117142GB-I00 [DeeLight project] by MCIN/AEI/10.13039/501100011033 and by “ERDF, a way of making Europe”, by the European Union). The work of Jordi Gené Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU. We would also like to thank Nufri (especially Santiago Salamero and Oriol Morreres) for their support during data acquisition, and Pieter van Dalfsen and Dirk de Hoog from Wageningen University & Research for additional data collection used in the case study., info:eu-repo/semantics/publishedVersion
IRTA Pubpro. Open Digital Archive
Publicación Capítulo o parte de un libro (BookPart)

How can precision agriculture prescription maps contribute to the 50% pesticide reduction goal of the farm-to-fork strategy in viticulture

  • Llorens, J.
  • Román, C.
  • Escolà, A.
  • Gené-Mola, Jordi
  • Martínez-Casasnovas, J.A.
The Farm-to-Fork (F2F) strategy of the European Green Deal demands a 50 % reduction in the use of pesticides by 2030. In this work, several Precision Agriculture solutions were used in a vineyard spray application and the results were compared to a conventional treatment. Three different treatments were applied with the farm air-blast sprayer in a commercial 2.5 ha vineyard. Treatment 1 was the conventional farm application of a uniform 450 L ha-1 volume rate. In Treatment 2, the decision support system (DSS) DOSA3D was used to decide an adjusted uniform volume rate of 430 L ha-1. Treatment 3 took advantage of several PA techniques: a) electronic canopy characterization using a mobile terrestrial laser scanner to map canopy geometric parameters; b) management zones delineation; c) DSS to decide two different dose rates to create a prescription map (low vigour: 340 L ha-1 and high vigour: 430 L ha-1); d) variable-rate technology to execute the prescription map implemented in the farm sprayer. The spray deposit on leaves was the parameter used to assess differences between treatments. The statistical analysis did not show significant differences between treatments in high vigour vines. However, in low vigour vines, the mean deposits of both uniform treatments increased and showed significant differences with the variable-rate treatment., This work was partly funded by the Spanish Ministry of Science, Innovation and Universities through the projects PAgFRUIT (RTI2018-094222-B-I00) and PAgPROTECT (PID2021-126648OB-I00) (by MCIN/AEI/10.13039/501100011033 and by “ERDF, a way of making Europe”, by the European Union). The work of Jordi Gené Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU., info:eu-repo/semantics/acceptedVersion
IRTA Pubpro. Open Digital Archive
Publicación Artículo científico (JournalArticle)

Simultaneous fruit detection and size estimation using multitask deep neural networks

  • Ferrer-Ferrer, Mar
  • Ruiz-Hidalgo, Javier
  • Gregorio, Eduard
  • Vilaplana, Verónica
  • Morros, Josep-Ramon
  • Gené-Mola, Jordi
The measurement of fruit size is of great interest to estimate the yield and predict the harvest resources in advance. This work proposes a novel technique for in-field apple detection and measurement based on Deep Neural Networks. The proposed framework was trained with RGB-D data and consists of an end-to-end multitask Deep Neural Network architecture specifically designed to perform the following tasks: 1) detection and segmentation of each fruit from its surroundings; 2) estimation of the diameter of each detected fruit. The methodology was tested with a total of 15,335 annotated apples at different growth stages, with diameters varying from 27 mm to 95 mm. Fruit detection results reported an F1-score for apple detection of 0.88 and a mean absolute error of diameter estimation of 5.64 mm. These are state-of-the-art results with the additional advantages of: a) using an end-to-end multitask trainable network; b) an efficient and fast inference speed; and c) being based on RGB-D data which can be acquired with affordable depth cameras. On the contrary, the main disadvantage is the need of annotating a large amount of data with fruit masks and diameter ground truth to train the model. Finally, a fruit visibility analysis showed an improvement in the prediction when limiting the measurement to apples above 65% of visibility (mean absolute error of 5.09 mm). This suggests that future works should develop a method for automatically identifying the most visible apples and discard the prediction of highly occluded fruits., This work was partly funded by the Departament de Recerca i Universitats de la Generalitat de Catalunya (grant 2021 LLAV 00088), the Spanish Ministry of Science, Innovation and Universities (grants RTI2018-094222-B-I00[PAgFRUIT project], PID2021-126648OB-I00 [PAgPROTECT project] and PID2020-117142 GB-I00 [DeeLight project] by MCIN/AEI/10.13039/501100011033 and by “ERDF, a way of making Europe”, by the European Union). The work of Jordi Gené Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU., info:eu-repo/semantics/publishedVersion
IRTA Pubpro. Open Digital Archive
Publicación Artículo científico (JournalArticle)

AmodalAppleSize_RGB-D dataset: RGB-D images of apple trees annotated with modal and amodal segmentation masks for fruit detection, visibility and size estimation

  • Gené-Mola, Jordi
  • Ferrer-Ferrer, Mar
  • Hemming, Jochen
  • van Dalfsen, Pieter
  • de Hoog, Dirk
  • Sanz-Cortiella, Ricardo
  • Rosell-Polo, Joan R.
  • Morros, Josep Ramon
  • Vilaplana, Verónica
  • Ruiz-Hidalgo, Javier
  • Gregorio, Eduard
The present dataset comprises a collection of RGB-D apple tree images that can be used to train and test computer vision-based fruit detection and sizing methods. This dataset encompasses two distinct sets of data obtained from a Fuji and an Elstar apple orchards. The Fuji apple orchard sub-set consists of 3925 RGB-D images containing a total of 15,335 apples annotated with both modal and amodal apple segmentation masks. Modal masks denote the visible portions of the apples, whereas amodal masks encompass both visible and occluded apple regions. Notably, this dataset is the first public resource to incorporate on-tree fruit amodal masks. This pioneering inclusion addresses a critical gap in existing datasets, enabling the development of robust automatic fruit sizing methods and accurate fruit visibility estimation, particularly in the presence of partial occlusions. Besides the fruit segmentation masks, the dataset also includes the fruit size (calliper) ground truth for each annotated apple. The second sub-set comprises 2731 RGB-D images capturing five Elstar apple trees at four distinct growth stages. This sub-set includes mean diameter information for each tree at every growth stage and serves as a valuable resource for evaluating fruit sizing methods trained with the first sub-set. The present data was employed in the research paper titled “Looking behind occlusions: a study on amodal segmentation for robust on-tree apple fruit size estimation” [1]., This work was partly funded by the Departament de Recerca i Universitats de la Generalitat de Catalunya (grant 2021 LLAV 00088), the Spanish Ministry of Science, Innovation and Universities (grants RTI2018-094222-B-I00 [PAgFRUIT project], PID2021-126648OB-I00 [PAgPROTECT project] and PID2020-117142GB-I00 [DeeLight project] by MCIN/AEI/10.13039/501100011033 and by “ERDF, a way of making Europe”, by the European Union). Data presented in this paper is also part of a Public Private Partnership project Precisie Tuinbouw, WP Fruit 4.0 (PPS KV 1604-025) and financed by Topsector Tuinbouw & Uitgangsmateriaal and various private companies. The work of Jordi Gené Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU. We would also like to thank Nufri (especially Santiago Salamero and Oriol Morreres) for their support during data acquisition., info:eu-repo/semantics/publishedVersion
IRTA Pubpro. Open Digital Archive
Publicación Artículo científico (JournalArticle)

A systematic analysis of scan matching techniques for localization in dense orchards

  • Guevara, Javier
  • Gené-Mola, Jordi
  • Gregorio, Eduard
  • Auat Cheein, Fernando A.
In recent years, different methods have been studied to determine machinery position within a grove, as an alternative for complementing GNSS (global navigation satellite system) information in cases where GNSS signal is occluded. Such a situation can be observed when agricultural machinery travels under dense foliage or on the slopes of mountains. Scan matching techniques arise as a possible solution for localizing the machinery, complementing the absence of the GNSS signal when necessary. However, since key points are difficult to obtain in heterogeneous, unstructured and non-rigid environments (such as orchard plants), the performance of scan matching techniques often decreases in agricultural environments. This paper suggests dividing the point clouds into horizontal and vertical segments to improve the performance of scan-matching methods in orchards. It also examines the best way for registered frames to overlap. We validate the analysis with extensive experimentation in a Fuji apple orchard. The results show that the cumulative localization error in scan matching techniques can be notoriously decreased with selective parts of the orchard, by up to 60%. The experimentation performed herein suggests that the proposed methodology can complement the GNSS navigation in a middle-long path., This work was partly funded by ANID FB0008, PIIC 030/2018 DGIIP-UTFSM Chile, the Secretaria d'Universitats i Recerca del Departament d'Empresa i Coneixement de la Generalitat de Catalunya (grant 2017 SGR 646), the Spanish Ministry of Science, Innovation and Universities (project RTI2018-094222-B-I00). This work is also part of the DIGIFRUIT project (grant TED2021-131871B-I00) funded by MICIU/AEI/10.13039/501100011033 and by the European Union NextGenerationEU/PRTR. The Spanish Ministry of Education is thanked for Mr. J. Gené's pre-doctoral fellowships (FPU15/03355)., info:eu-repo/semantics/publishedVersion
IRTA Pubpro. Open Digital Archive
Publicación Artículo científico (JournalArticle)

AKFruitYield: Modular benchmarking and video analysis software for Azure Kinect cameras for fruit size and fruit yield estimation in apple orchards

  • Miranda, Juan Carlos
  • Arnó, Jaume
  • Gené-Mola, Jordi
  • Fountas, Spyros
  • Gregorio, Eduard
AKFruitYield is a modular software that allows orchard data from RGB-D Azure Kinect cameras to be processed for fruit size and fruit yield estimation. Specifically, two modules have been developed: i) AK_SW_BENCHMARKER that makes it possible to apply different sizing algorithms and allometric yield prediction models to manually labelled color and depth tree images; and ii) AK_VIDEO_ANALYSER that analyses videos on which to automatically detect apples, estimate their size and predict yield at the plot or per hectare scale using the appropriate algorithms. Both modules have easy-to-use graphical interfaces and provide reports that can subsequently be used by other analysis tools., This work was partly funded by the Department of Research and Universities of the Generalitat de Catalunya (grants 2017 SGR 646) and by the Spanish Ministry of Science and Innovation/AEI/10.13039/501100011033/ERDF (grant RTI2018–094222-B-I00 [PAgFRUIT project] and PID2021–126648OB-I00 [PAgPROTECT project]). The Secretariat of Universities and Research of the Department of Business and Knowledge of the Generalitat de Catalunya and European Social Fund (ESF) are also thanked for financing Juan Carlos Miranda's pre-doctoral fellowship (2020 FI_B 00586). The work of Jordi Gené-Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU. The authors would also like to thank the Institut de Recerca i Tecnologia Agroalimentàries (IRTA) for allowing the use of their experimental fields, and in particular Dr. Luís Asín and Dr. Jaume Lordán who have contributed to the success of this work., info:eu-repo/semantics/publishedVersion
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

LiDAR-derived indices and their relationship with productivity and oil quality attributes in high-density olive orchards

  • Sandonis-Pozo, L.
  • Rufat, Josep
  • Pascual, M.
  • Villar, J.M.
  • Arnó, J.
  • Escolà, A.
  • Rosell-Polo, J.R.
  • Martínez-Casasnovas, J.A.
Optimising the leaf area index (LAI) and light exposure are key aspects to improve yield and quality in high-density olive orchards. Traditionally, these characteristics are estimated using ceptometers, but this method is both labour intensive and dependent on light conditions, with limitations in effectiveness for precision horticulture management purposes. This study investigates the potential of four LiDAR-derived indices to: i) estimate the LAI and canopy light interception under different fertigation treatments, and ii) to assess their impact on yield and oil quality throughout a growing season in a high-density olive orchard. The treatments included two nitrogen dose rates (N0: 0 kg ha-¹ and N50: 50 kg ha-¹) combined with full irrigation (FI) and deficit irrigation (DI). The results provide strong evidence for the effectiveness of these indices in monitoring canopy response to treatments over time. The N50 and FI treatments increased the LAI and yield but led to poorer light attenuation within the canopy and lower oil quality. In contrast, N0 and DI promoted more light-permeable canopies and improved oil quality, with higher phenolic content, better oxidative stability and improved organoleptic oil attributes. These indices can improve the understanding of canopy light environments, help to optimise management practices, and ultimately improve the sustainability and profitability of intensive olive cultivation., This work was carried out with data acquired in the project IRTA2012–00059-C02–01 funded by Spain’s National Institute for Agricultural and Food Research and Technology, as well as in the projects AGL2010–22304-C04–03 and AGL2013–48297-C2–2-R funded by the Spanish Ministry of Economy and Competitiveness. The study was carried out under the framework of the grants RTI2018–094222-B-I00 (PAgFRUIT project) and PID2021–126648OB-I00 (PAgPROTECT project) through MCIN/AEI/10.13039/501100011033 and the European Union’s “ERDF, a way of making Europe”., info:eu-repo/semantics/publishedVersion
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

AmodalAppleSize_RGB-D dataset: RGB-D images of apple trees annotated with modal and amodal segmentation masks for fruit detection, visibility and size estimation

  • Gené-Mola, Jordi
  • Ferrer-Ferrer, Mar
  • Hemming, Jochen
  • van Dalfsen, Pieter
  • de Hoog, Dirk
  • Sanz-Cortiella, Ricardo
  • Rosell-Polo, Joan R.
  • Morros, Josep Ramon
  • Vilaplana, Verónica
  • Ruiz-Hidalgo, Javier
  • Gregorio, Eduard
The present dataset comprises a collection of RGB-D apple tree images that can be used to train and test computer vision-based fruit detection and sizing methods. This dataset encompasses two distinct sets of data obtained from a Fuji and an Elstar apple orchards. The Fuji apple orchard sub-set consists of 3925 RGB-D images containing a total of 15,335 apples annotated with both modal and amodal apple segmentation masks. Modal masks denote the visible portions of the apples, whereas amodal masks encompass both visible and occluded apple regions. Notably, this dataset is the first public resource to incorporate on-tree fruit amodal masks. This pioneering inclusion addresses a critical gap in existing datasets, enabling the development of robust automatic fruit sizing methods and accurate fruit visibility estimation, particularly in the presence of partial occlusions. Besides the fruit segmentation masks, the dataset also includes the fruit size (calliper) ground truth for each annotated apple. The second sub-set comprises 2731 RGB-D images capturing five Elstar apple trees at four distinct growth stages. This sub-set includes mean diameter information for each tree at every growth stage and serves as a valuable resource for evaluating fruit sizing methods trained with the first sub-set. The present data was employed in the research paper titled “Looking behind occlusions: a study on amodal segmentation for robust on-tree apple fruit size estimation” [1]., This work was partly funded by the Departament de Recerca i Universitats de la Generalitat de Catalunya (grant 2021 LLAV 00088), the Spanish Ministry of Science, Innovation and Universities (grants RTI2018-094222-B-I00 [PAgFRUIT project], PID2021-126648OB-I00 [PAgPROTECT project] and PID2020-117142GB-I00 [DeeLight project] by MCIN/AEI/10.13039/501100011033 and by “ERDF, a way of making Europe”, by the European Union). Data presented in this paper is also part of a Public Private Partnership project Precisie Tuinbouw, WP Fruit 4.0 (PPS KV 1604-025) and financed by Topsector Tuinbouw & Uitgangsmateriaal and various private companies. The work of Jordi Gené Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU. We would also like to thank Nufri (especially Santiago Salamero and Oriol Morreres) for their support during data acquisition., info:eu-repo/semantics/publishedVersion
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

A systematic analysis of scan matching techniques for localization in dense orchards

  • Guevara, Javier
  • Gené-Mola, Jordi
  • Gregorio, Eduard
  • Auat Cheein, Fernando A.
In recent years, different methods have been studied to determine machinery position within a grove, as an alternative for complementing GNSS (global navigation satellite system) information in cases where GNSS signal is occluded. Such a situation can be observed when agricultural machinery travels under dense foliage or on the slopes of mountains. Scan matching techniques arise as a possible solution for localizing the machinery, complementing the absence of the GNSS signal when necessary. However, since key points are difficult to obtain in heterogeneous, unstructured and non-rigid environments (such as orchard plants), the performance of scan matching techniques often decreases in agricultural environments. This paper suggests dividing the point clouds into horizontal and vertical segments to improve the performance of scan-matching methods in orchards. It also examines the best way for registered frames to overlap. We validate the analysis with extensive experimentation in a Fuji apple orchard. The results show that the cumulative localization error in scan matching techniques can be notoriously decreased with selective parts of the orchard, by up to 60%. The experimentation performed herein suggests that the proposed methodology can complement the GNSS navigation in a middle-long path., This work was partly funded by ANID FB0008, PIIC 030/2018 DGIIP-UTFSM Chile, the Secretaria d'Universitats i Recerca del Departament d'Empresa i Coneixement de la Generalitat de Catalunya (grant 2017 SGR 646), the Spanish Ministry of Science, Innovation and Universities (project RTI2018-094222-B-I00). This work is also part of the DIGIFRUIT project (grant TED2021-131871B-I00) funded by MICIU/AEI/10.13039/501100011033 and by the European Union NextGenerationEU/PRTR. The Spanish Ministry of Education is thanked for Mr. J. Gené's pre-doctoral fellowships (FPU15/03355)., info:eu-repo/semantics/publishedVersion
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Video-based fruit detection and tracking: effects of scanning conditions on fruit load estimation

  • Felip-Pomés, Marc
  • Gené-Mola, Jordi
  • Arnó, Jaume
  • Lordan, Jaume
  • Ruiz-Hidalgo, Javier
  • Morros, Josep-Ramon
  • Arasanz-Malo, Miguel
  • Net-Barnés, Francesc
  • Gregorio, Eduard
Automated fruit counting and yield estimation systems are necessary for efficient orchard management. This study presents a computer vision system based on video multi-object-tracking for fruit load estimation in apple orchards and provides a comprehensive analysis of the system performance under diverse scanning conditions. The system integrates fruit detection, tracking, localization within orchard, and fruit load map generation. Experiments were carried out in an experimental apple orchard containing 420 apple trees. Data was collected with two different RGB-D sensors (Azure Kinect DK and ZED 2) at three different scanning distances (125 cm, 175 cm, and 225 cm) on two different dates prior to the harvest. Comparing the two evaluated sensors, Azure Kinect provided more consistent performance across different dates. Results also show that the longer scanning distance improves accuracy due to seeing the full tree view gives better fruit counts than close partial views. Between the two dates, best results were achieved near harvest due to fruit color at this stage, achieving a Mean Absolute Percentage Error (MAPE) of 6.91 % and a determination coefficient (R2) of 0.733 (using ZED2 sensor at 225 cm distance). Finally, a test comparing scanning from one or both sides of the tree row showed that bilateral scanning improved fruit load estimation at the stretch level by incorporating information from both sides of the canopy. The results of this work demonstrate the effectiveness of the video fruit tracking systems as a useful tool for automating fruit load estimation., This work was partly funded by the Department of Research and Universities of the Generalitat de Catalunya (grants 2017, SGR 646 and 2021 LLAV 00088), by the Spanish Ministry of Science and Innovation /AEI/10.13039/501100011033 / ERDF (grants RTI2018-094222-B-I00 [PAgFRUIT project], PID2021-126648OB-I00 [PAgPROTECT project], PID2024-161868OB-I00 [C3DRUM project], and PID2024-156495OR-C31 [YIELD4CAST]), by the Spanish Ministry of Science and Innovation / AEI/10.13039/501100011033 / European Union NextGeneration / PRTR (grant TED2021-131871B-I00 [DIGIFRUIT project]), and by the Agrotecnio-IRTA Joint Projects Call 2025 (PhotonFruit project). The work of Jordi Gené-Mola was partially supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU., info:eu-repo/semantics/publishedVersion
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

AKFruitYield: Modular benchmarking and video analysis software for Azure Kinect cameras for fruit size and fruit yield estimation in apple orchards

  • Miranda, Juan Carlos
  • Arnó, Jaume
  • Gené-Mola, Jordi
  • Fountas, Spyros
  • Gregorio, Eduard
AKFruitYield is a modular software that allows orchard data from RGB-D Azure Kinect cameras to be processed for fruit size and fruit yield estimation. Specifically, two modules have been developed: i) AK_SW_BENCHMARKER that makes it possible to apply different sizing algorithms and allometric yield prediction models to manually labelled color and depth tree images; and ii) AK_VIDEO_ANALYSER that analyses videos on which to automatically detect apples, estimate their size and predict yield at the plot or per hectare scale using the appropriate algorithms. Both modules have easy-to-use graphical interfaces and provide reports that can subsequently be used by other analysis tools., This work was partly funded by the Department of Research and Universities of the Generalitat de Catalunya (grants 2017 SGR 646) and by the Spanish Ministry of Science and Innovation/AEI/10.13039/501100011033/ERDF (grant RTI2018–094222-B-I00 [PAgFRUIT project] and PID2021–126648OB-I00 [PAgPROTECT project]). The Secretariat of Universities and Research of the Department of Business and Knowledge of the Generalitat de Catalunya and European Social Fund (ESF) are also thanked for financing Juan Carlos Miranda's pre-doctoral fellowship (2020 FI_B 00586). The work of Jordi Gené-Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU. The authors would also like to thank the Institut de Recerca i Tecnologia Agroalimentàries (IRTA) for allowing the use of their experimental fields, and in particular Dr. Luís Asín and Dr. Jaume Lordán who have contributed to the success of this work., info:eu-repo/semantics/publishedVersion
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Assessing automatic data processing algorithms for RGB-D cameras to predict fruit size and weight in apples

  • Miranda, Juan C.
  • Arnó, Jaume
  • Gené-Mola, Jordi
  • Lordan, Jaume
  • Asín, Luis
  • Gregorio, Eduard
Data acquired using an RGB-D Azure Kinect DK camera were used to assess different automatic algorithms to estimate the size, and predict the weight of non-occluded and occluded apples. The programming of the algorithms included: (i) the extraction of images of regions of interest (ROI) using manual delimitation of bounding boxes or binary masks; (ii) estimating the lengths of the major and minor geometric axes for the purpose of apple sizing; and (iii) predicting the final weight by allometric modelling. In addition to the use of bounding boxes, the algorithms also allowed other post-mask settings (circles, ellipses and rotated rectangles) to be implemented, and different depth options (distance between the RGB-D camera and the fruits detected) for subsequent sizing through the application of the thin lens theory. Both linear and nonlinear allometric models demonstrated the ability to predict apple weight with a high degree of accuracy (R2 greater than 0.942 and RMSE < 16 g). With respect to non-occluded apples, the best weight predictions were achieved using a linear allometric model including both the major and minor axes of the apples as predictors. The mean absolute percentage error (MAPE) ranged from 5.1% to 5.7% with respective RMSE of 11.09 g and 13.02 g, depending to whether circles, ellipses, or bounding boxes were used to adjust fruit shape. The results were therefore promising and open up the possibility of implementing reliable in-field apple measurements in real time. Importantly, final weight prediction error and intermediate size estimation errors (from sizing algorithms) interact but in a way that is not easily quantifiable when weight allometric models with implicit prediction error are used. In addition, allometric models should be reviewed when applied to other apple cultivars, fruit development stages or even for different fruit growth conditions depending on canopy management., This work was partly funded by the Department of Research and Universities of the Generalitat de Catalunya (grants 2017, SGR 646 and 2021 LLAV 00088), by the Spanish Ministry of Science and Innovation / AEI/10.13039/501100011033 / ERDF (grants RTI2018-094222-B-I00 [PAgFRUIT project], PID2021-126648OB-I00 [PAgPROTECT project]) and by the Spanish Ministry of Science and Innovation / AEI/10.13039/501100011033 / European Union NextGeneration / PRTR (grantTED2021-131871B-I00 [DIGIFRUIT project]). We would also like to thank the Secretariat of Universities and Research of the Department of Business and Knowledge of the Generalitat de Catalunya and the European Social Fund (ESF) for financing Juan Carlos Miranda’s pre-doctoral fellowship (2020 FI_B 00586). The work of Jordi Gené-Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU., info:eu-repo/semantics/publishedVersion
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Simultaneous fruit detection and size estimation using multitask deep neural networks

  • Ferrer-Ferrer, Mar
  • Ruiz-Hidalgo, Javier
  • Gregorio, Eduard
  • Vilaplana, Verónica
  • Morros, Josep-Ramon
  • Gené-Mola, Jordi
The measurement of fruit size is of great interest to estimate the yield and predict the harvest resources in advance. This work proposes a novel technique for in-field apple detection and measurement based on Deep Neural Networks. The proposed framework was trained with RGB-D data and consists of an end-to-end multitask Deep Neural Network architecture specifically designed to perform the following tasks: 1) detection and segmentation of each fruit from its surroundings; 2) estimation of the diameter of each detected fruit. The methodology was tested with a total of 15,335 annotated apples at different growth stages, with diameters varying from 27 mm to 95 mm. Fruit detection results reported an F1-score for apple detection of 0.88 and a mean absolute error of diameter estimation of 5.64 mm. These are state-of-the-art results with the additional advantages of: a) using an end-to-end multitask trainable network; b) an efficient and fast inference speed; and c) being based on RGB-D data which can be acquired with affordable depth cameras. On the contrary, the main disadvantage is the need of annotating a large amount of data with fruit masks and diameter ground truth to train the model. Finally, a fruit visibility analysis showed an improvement in the prediction when limiting the measurement to apples above 65% of visibility (mean absolute error of 5.09 mm). This suggests that future works should develop a method for automatically identifying the most visible apples and discard the prediction of highly occluded fruits., This work was partly funded by the Departament de Recerca i Universitats de la Generalitat de Catalunya (grant 2021 LLAV 00088), the Spanish Ministry of Science, Innovation and Universities (grants RTI2018-094222-B-I00[PAgFRUIT project], PID2021-126648OB-I00 [PAgPROTECT project] and PID2020-117142 GB-I00 [DeeLight project] by MCIN/AEI/10.13039/501100011033 and by “ERDF, a way of making Europe”, by the European Union). The work of Jordi Gené Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU., info:eu-repo/semantics/publishedVersion
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Fruit sizing using AI: A review of methods and challenges

  • Miranda, Juan C.
  • Gené-Mola, Jordi
  • Zude-Sasse, Manuela
  • Tsoulias, Nikos
  • Escolà, Alexandre
  • Arnó, Jaume
  • Rosell-Polo, Joan R.
  • Sanz-Cortiella, Ricardo
  • Martínez-Casasnovas, José A.
  • Gregorio, Eduard
Fruit size at harvest is an economically important variable for high-quality table fruit production in orchards and vineyards. In addition, knowing the number and size of the fruit on the tree is essential in the framework of precise production, harvest, and postharvest management. A prerequisite for analysis of fruit in a real-world environment is the detection and segmentation from background signal. In the last five years, deep learning convolutional neural network have become the standard method for automatic fruit detection, achieving F1-scores higher than 90 %, as well as real-time processing speeds. At the same time, different methods have been developed for, mainly, fruit size and, more rarely, fruit maturity estimation from 2D images and 3D point clouds. These sizing methods are focused on a few species like grape, apple, citrus, and mango, resulting in mean absolute error values of less than 4 mm in apple fruit. This review provides an overview of the most recent methodologies developed for in-field fruit detection/counting and sizing as well as few upcoming examples of maturity estimation. Challenges, such as sensor fusion, highly varying lighting conditions, occlusions in the canopy, shortage of public fruit datasets, and opportunities for research transfer, are discussed., This work was partly funded by the Department of Research and Universities of the Generalitat de Catalunya (grants 2017 SGR 646 and 2021 LLAV 00088) and by the Spanish Ministry of Science and Innovation / AEI/10.13039/501100011033 / FEDER (grants RTI2018-094222-B-I00 [PAgFRUIT project] and PID2021-126648OB-I00 [PAgPROTECT project]). The Secretariat of Universities and Research of the Department of Business and Knowledge of the Generalitat de Catalunya and European Social Fund (ESF) are also thanked for financing Juan Carlos Miranda’s pre-doctoral fellowship (2020 FI_B 00586). The work of Jordi Gené-Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU., info:eu-repo/semantics/publishedVersion
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Looking behind occlusions: A study on amodal segmentation for robust on-tree apple fruit size estimation

  • Gené-Mola, Jordi
  • Ferrer-Ferrer, Mar
  • Gregorio, Eduard
  • Blok, Pieter M.
  • Hemming, Jochen
  • Morros, Josep-Ramon
  • Rosell-Polo, Joan R.
  • Vilaplana, Verónica
  • Ruiz-Hidalgo, Javier
The detection and sizing of fruits with computer vision methods is of interest because it provides relevant information to improve the management of orchard farming. However, the presence of partially occluded fruits limits the performance of existing methods, making reliable fruit sizing a challenging task. While previous fruit segmentation works limit segmentation to the visible region of fruits (known as modal segmentation), in this work we propose an amodal segmentation algorithm to predict the complete shape, which includes its visible and occluded regions. To do so, an end-to-end convolutional neural network (CNN) for simultaneous modal and amodal instance segmentation was implemented. The predicted amodal masks were used to estimate the fruit diameters in pixels. Modal masks were used to identify the visible region and measure the distance between the apples and the camera using the depth image. Finally, the fruit diameters in millimetres (mm) were computed by applying the pinhole camera model. The method was developed with a Fuji apple dataset consisting of 3925 RGB-D images acquired at different growth stages with a total of 15,335 annotated apples, and was subsequently tested in a case study to measure the diameter of Elstar apples at different growth stages. Fruit detection results showed an F1-score of 0.86 and the fruit diameter results reported a mean absolute error (MAE) of 4.5 mm and R2 = 0.80 irrespective of fruit visibility. Besides the diameter estimation, modal and amodal masks were used to automatically determine the percentage of visibility of measured apples. This feature was used as a confidence value, improving the diameter estimation to MAE = 2.93 mm and R2 = 0.91 when limiting the size estimation to fruits detected with a visibility higher than 60%. The main advantages of the present methodology are its robustness for measuring partially occluded fruits and the capability to determine the visibility percentage. The main limitation is that depth images were generated by means of photogrammetry methods, which limits the efficiency of data acquisition. To overcome this limitation, future works should consider the use of commercial RGB-D sensors. The code and the dataset used to evaluate the method have been made publicly available at https://github.com/GRAP-UdL-AT/Amodal_Fruit_Sizing, This work was partly funded by the Departament de Recerca i Universitats de la Generalitat de Catalunya (grant 2021 LLAV 00088), the Spanish Ministry of Science, Innovation and Universities (grants RTI2018-094222-B-I00 [PAgFRUIT project], PID2021-126648OB-I00 [PAgPROTECT project] and PID2020-117142GB-I00 [DeeLight project] by MCIN/AEI/10.13039/501100011033 and by “ERDF, a way of making Europe”, by the European Union). The work of Jordi Gené Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU. We would also like to thank Nufri (especially Santiago Salamero and Oriol Morreres) for their support during data acquisition, and Pieter van Dalfsen and Dirk de Hoog from Wageningen University & Research for additional data collection used in the case study., info:eu-repo/semantics/publishedVersion
Recercat. Dipósit de la Recerca de Catalunya
Publicación Capítulo o parte de un libro (BookPart)

How can precision agriculture prescription maps contribute to the 50% pesticide reduction goal of the farm-to-fork strategy in viticulture

  • Llorens, J.
  • Román, C.
  • Escolà, A.
  • Gené-Mola, Jordi
  • Martínez-Casasnovas, J.A.
The Farm-to-Fork (F2F) strategy of the European Green Deal demands a 50 % reduction in the use of pesticides by 2030. In this work, several Precision Agriculture solutions were used in a vineyard spray application and the results were compared to a conventional treatment. Three different treatments were applied with the farm air-blast sprayer in a commercial 2.5 ha vineyard. Treatment 1 was the conventional farm application of a uniform 450 L ha-1 volume rate. In Treatment 2, the decision support system (DSS) DOSA3D was used to decide an adjusted uniform volume rate of 430 L ha-1. Treatment 3 took advantage of several PA techniques: a) electronic canopy characterization using a mobile terrestrial laser scanner to map canopy geometric parameters; b) management zones delineation; c) DSS to decide two different dose rates to create a prescription map (low vigour: 340 L ha-1 and high vigour: 430 L ha-1); d) variable-rate technology to execute the prescription map implemented in the farm sprayer. The spray deposit on leaves was the parameter used to assess differences between treatments. The statistical analysis did not show significant differences between treatments in high vigour vines. However, in low vigour vines, the mean deposits of both uniform treatments increased and showed significant differences with the variable-rate treatment., This work was partly funded by the Spanish Ministry of Science, Innovation and Universities through the projects PAgFRUIT (RTI2018-094222-B-I00) and PAgPROTECT (PID2021-126648OB-I00) (by MCIN/AEI/10.13039/501100011033 and by “ERDF, a way of making Europe”, by the European Union). The work of Jordi Gené Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU., info:eu-repo/semantics/acceptedVersion