Skip to main content

ESTRECHANDO LA BRECHA ENTRE LA ESTADISTICA Y LA CIENCIA DE DATOS

MTM2017-88142-P

•
Funding agency name Agencia Estatal de Investigación
Funding agency acronym AEI
Program Programa Estatal de Fomento de la Investigación Científica y Técnica de Excelencia
Subprogram Subprograma Estatal de Generación de Conocimiento
Call Proyectos de I+D (Excelencia / Generación de Conocimiento)
Call year 2017
State Plan 2017-2020 Plan Estatal de Investigación Científica y Técnica y de Innovación
Beneficiary centre UNIVERSITAT POLITECNICA DE CATALUNYA

Resultados relacionados

Resultados totales (Incluyendo duplicados): 9
Encontrada(s) 1 página(s)

UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Understanding complex predictive models with ghost variables

  • Delicado Useros, Pedro Francisco|||0000-0003-3933-4852
  • Peña Sanchez de Rivera, Daniel
The version of record of this article, first published in Test, is available online at Publisher’s website: http://dx.doi.org/10.1007/s11749-022-00826-x, Framed in the literature on Interpretable Machine Learning, we propose a new procedure to assign a measure of relevance to each explanatory variable in a complex predictive model. We assume that we have a training set to fit the model and a test set to check its out-of-sample performance. We propose to measure the individual relevance of each variable by comparing the predictions of the model in the test set with those obtained when the variable of interest is substituted (in the test set) by its ghost variable, defined as the prediction of this variable by using the rest of explanatory variables. In linear models it is shown that, on the one hand, the proposed measure gives similar results to leave-one-covariate-out (loco, with a lowest computational cost) and outperforms random permutations, and on the other hand, it is strongly related to the usual F-statistic measuring the significance of a variable. In nonlinear predictive models (as neural networks or random forests) the proposed measure shows the relevance of the variables in an efficient way, as shown by a simulation study comparing ghost variables with other alternative methods (including loco and random permutations, and also knockoff variables and estimated conditional distributions). Finally, we study the joint relevance of the variables by defining the relevance matrix as the covariance matrix of the vectors of effects on predictions when using every ghost variable. Our proposal is illustrated with simulated examples and the analysis of a large real data set., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Functional regression on remote sensing data in oceanography

  • Acar Denizli, Nihan|||0000-0002-0012-8632
  • Delicado Useros, Pedro Francisco|||0000-0003-3933-4852
  • Basarir, Gülay
  • Caballero de Frutos, Isabel
The final publication is available at link.springer.com, The aim of this study is to propose the use of a functional data analysis approach as an alternative to the classical statistical methods most commonly used in oceanography and water quality management. In particular we consider the prediction of total suspended solids (TSS) based on remote sensing (RS) data. For this purpose several functional linear regression models and classical non-functional regression models are applied to 10 years of RS data obtained from medium resolution imaging spectrometer sensor to predict the TSS concentration in the coastal zone of the Guadalquivir estuary. The results of functional and classical approaches are compared in terms of their mean square prediction error values and the superiority of the functional models is established. A simulation study has been designed in order to support these findings and to determine the best prediction model for the TSS parameter in more general contexts., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Estimating heterogeneous wildfire effects using synthetic controls and satellite remote sensing

  • Serra Burriel, Feliu
  • Delicado Useros, Pedro Francisco|||0000-0003-3933-4852
  • Prata, Andrew T.
  • Cucchietti, Fernando|||0000-0002-9027-1263
Wildfires have become one of the biggest natural hazards for environments worldwide. The effects of wildfires are heterogeneous, meaning that the magnitude of their effects depends on many factors such as geographical region, climate and land cover/vegetation type. Yet, which areas are more affected by these events remains unclear. Here we present a novel application of the Generalized Synthetic Control (GSC) method that enables quantification and prediction of vegetation changes due to wildfires through a time-series analysis of in situ and satellite remote sensing data. We apply this method to medium to large wildfires (> 1000 acres) in California throughout a time-span of two decades (1996–2016). The method's ability for estimating counterfactual vegetation characteristics for burned regions is explored in order to quantify abrupt system changes. We find that the GSC method is better at predicting vegetation changes than the more traditional approach of using nearby regions to assess wildfire impacts. We evaluate the GSC method by comparing its predictions of spectral vegetation indices to observations during pre-wildfire periods and find improvements in correlation coefficient from R2 = 0.66 to R2 = 0.93 in Normalized Difference Vegetation Index (NDVI), from R2 = 0.48 to R2 = 0.81 for Normalized Burn Ratio (NBR), and from R2 = 0.49 to R2 = 0.85 for Normalized Difference Moisture Index (NDMI). Results show greater changes in NDVI, NBR, and NDMI post-fire on regions classified as having a lower Burning Index. We find that on average, wildfires cause a 25% initial decrease in the vegetation index (NDVI) and a larger than 80% drop in wetness indices (NBR and NDMI) after they occur. The GSC method also reveals that wildfire effects on vegetation can last for more than a decade post-wildfire, and in some cases never return to their previous vegetation cycles within our study period. We also find that the dynamical effects vary across regions and have an impact on seasonal cycles of vegetation in later years. Lastly, we discuss the usefulness of using GSC in remote sensing analyses., F. S.-B. would also like to thank the Barcelona Supercomputing Center for the Severo Ochoa Mobility Grant, and Delicado would like to thank the Spanish Ministerio de Ciencia e Innovación for the grant MTM2017-88142-P, and A. T. P. acknowledges funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement H2020-MSCA-COFUND-2016-754433., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Wildfires vegetation recovery through satellite remote sensing and functional data analysis

  • Serra Burriel, Feliu
  • Delicado Useros, Pedro Francisco|||0000-0003-3933-4852
  • Cucchietti, Fernando|||0000-0002-9027-1263
In recent years, wildfires have caused havoc across the world, which are especially aggravated in certain regions due to climate change. Remote sensing has become a powerful tool for monitoring fires, as well as for measuring their effects on vegetation over the following years. We aim to explain the dynamics of wildfires’ effects on a vegetation index (previously estimated by causal inference through synthetic controls) from pre-wildfire available information (mainly proceeding from satellites). For this purpose, we use regression models from Functional Data Analysis, where wildfire effects are considered functional responses, depending on elapsed time after each wildfire, while pre-wildfire information acts as scalar covariates. Our main findings show that vegetation recovery after wildfires is a slow process, affected by many pre-wildfire conditions, among which the richness and diversity of vegetation is one of the best predictors for the recovery., Serra-Burriel would like to thank the Barcelona Supercomputing Center for the Severo Ochoa Mobility Grant, and Delicado would like to thank the Spanish Ministerio de Ciencia e Innovación for the grant MTM2017-88142-P, Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Wildfires vegetation recovery through satellite remote sensing and functional data analysis

  • Serra Burriel, Feliu
  • Delicado Useros, Pedro Francisco
  • Cucchietti, Fernando
In recent years, wildfires have caused havoc across the world, which are especially aggravated in certain regions due to climate change. Remote sensing has become a powerful tool for monitoring fires, as well as for measuring their effects on vegetation over the following years. We aim to explain the dynamics of wildfires’ effects on a vegetation index (previously estimated by causal inference through synthetic controls) from pre-wildfire available information (mainly proceeding from satellites). For this purpose, we use regression models from Functional Data Analysis, where wildfire effects are considered functional responses, depending on elapsed time after each wildfire, while pre-wildfire information acts as scalar covariates. Our main findings show that vegetation recovery after wildfires is a slow process, affected by many pre-wildfire conditions, among which the richness and diversity of vegetation is one of the best predictors for the recovery., Serra-Burriel would like to thank the Barcelona Supercomputing Center for the Severo Ochoa Mobility Grant, and Delicado would like to thank the Spanish Ministerio de Ciencia e Innovación for the grant MTM2017-88142-P, Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Estimating heterogeneous wildfire effects using synthetic controls and satellite remote sensing

  • Serra Burriel, Feliu
  • Delicado Useros, Pedro Francisco
  • Prata, Andrew T.
  • Cucchietti, Fernando
Wildfires have become one of the biggest natural hazards for environments worldwide. The effects of wildfires are heterogeneous, meaning that the magnitude of their effects depends on many factors such as geographical region, climate and land cover/vegetation type. Yet, which areas are more affected by these events remains unclear. Here we present a novel application of the Generalized Synthetic Control (GSC) method that enables quantification and prediction of vegetation changes due to wildfires through a time-series analysis of in situ and satellite remote sensing data. We apply this method to medium to large wildfires (> 1000 acres) in California throughout a time-span of two decades (1996–2016). The method's ability for estimating counterfactual vegetation characteristics for burned regions is explored in order to quantify abrupt system changes. We find that the GSC method is better at predicting vegetation changes than the more traditional approach of using nearby regions to assess wildfire impacts. We evaluate the GSC method by comparing its predictions of spectral vegetation indices to observations during pre-wildfire periods and find improvements in correlation coefficient from R2 = 0.66 to R2 = 0.93 in Normalized Difference Vegetation Index (NDVI), from R2 = 0.48 to R2 = 0.81 for Normalized Burn Ratio (NBR), and from R2 = 0.49 to R2 = 0.85 for Normalized Difference Moisture Index (NDMI). Results show greater changes in NDVI, NBR, and NDMI post-fire on regions classified as having a lower Burning Index. We find that on average, wildfires cause a 25% initial decrease in the vegetation index (NDVI) and a larger than 80% drop in wetness indices (NBR and NDMI) after they occur. The GSC method also reveals that wildfire effects on vegetation can last for more than a decade post-wildfire, and in some cases never return to their previous vegetation cycles within our study period. We also find that the dynamical effects vary across regions and have an impact on seasonal cycles of vegetation in later years. Lastly, we discuss the usefulness of using GSC in remote sensing analyses., F. S.-B. would also like to thank the Barcelona Supercomputing Center for the Severo Ochoa Mobility Grant, and Delicado would like to thank the Spanish Ministerio de Ciencia e Innovación for the grant MTM2017-88142-P, and A. T. P. acknowledges funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement H2020-MSCA-COFUND-2016-754433., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Understanding complex predictive models with ghost variables

  • Delicado Useros, Pedro Francisco
  • Peña Sanchez de Rivera, Daniel
The version of record of this article, first published in Test, is available online at Publisher’s website: http://dx.doi.org/10.1007/s11749-022-00826-x, Framed in the literature on Interpretable Machine Learning, we propose a new procedure to assign a measure of relevance to each explanatory variable in a complex predictive model. We assume that we have a training set to fit the model and a test set to check its out-of-sample performance. We propose to measure the individual relevance of each variable by comparing the predictions of the model in the test set with those obtained when the variable of interest is substituted (in the test set) by its ghost variable, defined as the prediction of this variable by using the rest of explanatory variables. In linear models it is shown that, on the one hand, the proposed measure gives similar results to leave-one-covariate-out (loco, with a lowest computational cost) and outperforms random permutations, and on the other hand, it is strongly related to the usual F-statistic measuring the significance of a variable. In nonlinear predictive models (as neural networks or random forests) the proposed measure shows the relevance of the variables in an efficient way, as shown by a simulation study comparing ghost variables with other alternative methods (including loco and random permutations, and also knockoff variables and estimated conditional distributions). Finally, we study the joint relevance of the variables by defining the relevance matrix as the covariance matrix of the vectors of effects on predictions when using every ghost variable. Our proposal is illustrated with simulated examples and the analysis of a large real data set., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Functional regression on remote sensing data in oceanography

  • Acar Denizli, Nihan
  • Delicado Useros, Pedro Francisco
  • Basarir, Gülay
  • Caballero de Frutos, Isabel
The final publication is available at link.springer.com, The aim of this study is to propose the use of a functional data analysis approach as an alternative to the classical statistical methods most commonly used in oceanography and water quality management. In particular we consider the prediction of total suspended solids (TSS) based on remote sensing (RS) data. For this purpose several functional linear regression models and classical non-functional regression models are applied to 10 years of RS data obtained from medium resolution imaging spectrometer sensor to predict the TSS concentration in the coastal zone of the Guadalquivir estuary. The results of functional and classical approaches are compared in terms of their mean square prediction error values and the superiority of the functional models is established. A simulation study has been designed in order to support these findings and to determine the best prediction model for the TSS parameter in more general contexts., Peer Reviewed, Postprint (author's final draft)
Dipòsit Digital de Documents de la UAB
Publication Artículo científico (JournalArticle)

Identifying and Classifying Aberrant Response Patterns Through Functional Data Analysis

  • Doval Dieguez, Eduardo|||0000-0001-8416-160X
  • Delicado, Pedro
We propose new methods for identifying and classifying aberrant response patterns (ARPs) by means of functional data analysis. These methods take the person response function (PRF) of an individual and compare it with the pattern that would correspond to a generic individual of the same ability according to the item-person response surface. ARPs correspond to atypical difference functions. The ARP classification is done with functional data clustering applied to the PRFs identified as ARP. We apply these methods to two sets of simulated data (the first is used to illustrate the ARP identification methods and the second demonstrates classification of the response patterns flagged as ARP) and a real data set (a Grade 12 science assessment test, SAT, with 32 items answered by 600 examinees). For comparative purposes, ARPs are also identified with three nonparametric person-fit indices (Ht, Modified Caution Index, and ZU3). Our results indicate that the ARP detection ability of one of our proposed methods is comparable to that of person-fit indices. Moreover, the proposed classification methods enable ARP associated with either spuriously low or spuriously high scores to be distinguished.