Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 6
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Concentration of posterior model probabilities and normalized L0 criteria

  • Rossell Ribera, David
We study frequentist properties of Bayesian and L0 model selection, with a focus on (potentially non-linear) high-dimensional regression. We propose a construction to study how posterior probabilities and normalized L0 criteria concentrate on the (Kullback-Leibler) optimal model and other subsets of the model space. When such concentration occurs, one also bounds the frequentist probabilities of selecting the correct model, type I and type II errors. These results hold generally, and help validate the use of posterior probabilities and L0 criteria to control frequentist error probabilities associated to model selection and hypothesis tests. Regarding regression, we help understand the effect of the sparsity imposed by the prior or the L0 penalty, and of problem characteristics such as the sample size, signal-to-noise, dimension and true sparsity. A particular finding is that one may use less sparse formulations than would be asymptotically optimal, but still attain consistency and often also significantly better finite-sample performance. We also prove new results related to misspecifying the mean or covariance structures, and give tighter rates for certain non-local priors than currently available., DR was partially funded by the Europa Excelencia grant EUR2020-112096, NIH grant R01 CA158113-01, Ramón y Cajal Fellowship RYC-2015-18544, Plan Estatal PGC2018-101643-B-I00 and Ayudas Fundación BBVA a equipos de investigación científica en Big Data 2017.
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

General Bayesian loss function selection and the use of improper models

  • Jewson, Jack
  • Rossell Ribera, David
Statisticians often face the choice between using probability models or a paradigm defined by minimising a loss function. Both approaches are useful and, if the loss can be re-cast into a proper probability model, there are many tools to decide which model or loss is more appropriate for the observed data, in the sense of explaining the data's nature. However, when the loss leads to an improper model, there are no principled ways to guide this choice. We address this task by combining the Hyvärinen score, which naturally targets infinitesimal relative probabilities, and general Bayesian updating, which provides a unifying framework for inference on losses and models. Specifically we propose the ℋ-score, a general Bayesian selection criterion and prove that it consistently selects the (possibly improper) model closest to the data-generating truth in Fisher's divergence. We also prove that an associated ℋ-posterior consistently learns optimal hyper-parameters featuring in loss functions, including a challenging tempering parameter in generalised Bayesian inference. As salient examples, we consider robust regression and non-parametric density estimation where popular loss functions define improper models for the data and hence cannot be dealt with using standard model selection tools. These examples illustrate advantages in robustness-efficiency trade-offs and enable Bayesian inference for kernel density estimation, opening a new avenue for Bayesian non-parametrics., Jack Jewson and David Rossell were partially funded by the Ayudas Fundación BBVA a Equipos de Investigación Cientifica 2017 and Proyectos de Investigación Científica en Matemáticas 2021, and Government of Spain's Plan Nacional PGC2018-101643-B-I00 grants. David Rossell was also partially funded by the Europa Excelencia grant EUR2020-112096 and Ramón y Cajal Fellowship RYC-2015-18544.
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Approximate Laplace approximations for scalable model selection

  • Rossell Ribera, David
  • Abril Pla, Oriol
  • Bhattacharya, Anirban
Includes supplementary materials for the online appendix., We propose the approximate Laplace approximation (ALA) to evaluate integrated likelihoods, a bottleneck in Bayesian model selection. The Laplace approximation (LA) is a popular tool that speeds up such computation and equips strong model selection properties. However, when the sample size is large or one considers many models the cost of the required optimizations becomes impractical. ALA reduces the cost to that of solving a least-squares problem for each model. Further, it enables efficient computation across models such as sharing pre-computed sufficient statistics and certain operations in matrix decompositions. We prove that in generalized (possibly non-linear) models ALA achieves a strong form of model selection consistency for a suitably-defined optimal model, at the same functional rates as exact computation. We consider fixed- and high-dimensional problems, group and hierarchical constraints, and the possibility that all models are misspecified. We also obtain ALA rates for Gaussian regression under non-local priors, an important example where the LA can be costly and does not consistently estimate the integrated likelihood. Our examples include non-linear regression, logistic, Poisson and survival models. We implement the methodology in the R package mombf., Spanish Government grants Europa Excelencia, Grant/Award Number: EUR2020-112096, RYC-2015-18544 and PGC2018-101643-B-I00; NIH, Grant/Award Number: R01 CA158113DMS-01.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Concentration of posterior model probabilities and normalized L0 criteria

  • Rossell Ribera, David
We study frequentist properties of Bayesian and L0 model selection, with a focus on (potentially non-linear) high-dimensional regression. We propose a construction to study how posterior probabilities and normalized L0 criteria concentrate on the (Kullback-Leibler) optimal model and other subsets of the model space. When such concentration occurs, one also bounds the frequentist probabilities of selecting the correct model, type I and type II errors. These results hold generally, and help validate the use of posterior probabilities and L0 criteria to control frequentist error probabilities associated to model selection and hypothesis tests. Regarding regression, we help understand the effect of the sparsity imposed by the prior or the L0 penalty, and of problem characteristics such as the sample size, signal-to-noise, dimension and true sparsity. A particular finding is that one may use less sparse formulations than would be asymptotically optimal, but still attain consistency and often also significantly better finite-sample performance. We also prove new results related to misspecifying the mean or covariance structures, and give tighter rates for certain non-local priors than currently available., DR was partially funded by the Europa Excelencia grant EUR2020-112096, NIH grant R01 CA158113-01, Ramón y Cajal Fellowship RYC-2015-18544, Plan Estatal PGC2018-101643-B-I00 and Ayudas Fundación BBVA a equipos de investigación científica en Big Data 2017.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Approximate Laplace approximations for scalable model selection

  • Rossell Ribera, David
  • Abril Pla, Oriol
  • Bhattacharya, Anirban
Includes supplementary materials for the online appendix., We propose the approximate Laplace approximation (ALA) to evaluate integrated likelihoods, a bottleneck in Bayesian model selection. The Laplace approximation (LA) is a popular tool that speeds up such computation and equips strong model selection properties. However, when the sample size is large or one considers many models the cost of the required optimizations becomes impractical. ALA reduces the cost to that of solving a least-squares problem for each model. Further, it enables efficient computation across models such as sharing pre-computed sufficient statistics and certain operations in matrix decompositions. We prove that in generalized (possibly non-linear) models ALA achieves a strong form of model selection consistency for a suitably-defined optimal model, at the same functional rates as exact computation. We consider fixed- and high-dimensional problems, group and hierarchical constraints, and the possibility that all models are misspecified. We also obtain ALA rates for Gaussian regression under non-local priors, an important example where the LA can be costly and does not consistently estimate the integrated likelihood. Our examples include non-linear regression, logistic, Poisson and survival models. We implement the methodology in the R package mombf., Spanish Government grants Europa Excelencia, Grant/Award Number: EUR2020-112096, RYC-2015-18544 and PGC2018-101643-B-I00; NIH, Grant/Award Number: R01 CA158113DMS-01.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

General Bayesian loss function selection and the use of improper models

  • Jewson, Jack
  • Rossell Ribera, David
Statisticians often face the choice between using probability models or a paradigm defined by minimising a loss function. Both approaches are useful and, if the loss can be re-cast into a proper probability model, there are many tools to decide which model or loss is more appropriate for the observed data, in the sense of explaining the data's nature. However, when the loss leads to an improper model, there are no principled ways to guide this choice. We address this task by combining the Hyvärinen score, which naturally targets infinitesimal relative probabilities, and general Bayesian updating, which provides a unifying framework for inference on losses and models. Specifically we propose the ℋ-score, a general Bayesian selection criterion and prove that it consistently selects the (possibly improper) model closest to the data-generating truth in Fisher's divergence. We also prove that an associated ℋ-posterior consistently learns optimal hyper-parameters featuring in loss functions, including a challenging tempering parameter in generalised Bayesian inference. As salient examples, we consider robust regression and non-parametric density estimation where popular loss functions define improper models for the data and hence cannot be dealt with using standard model selection tools. These examples illustrate advantages in robustness-efficiency trade-offs and enable Bayesian inference for kernel density estimation, opening a new avenue for Bayesian non-parametrics., Jack Jewson and David Rossell were partially funded by the Ayudas Fundación BBVA a Equipos de Investigación Cientifica 2017 and Proyectos de Investigación Científica en Matemáticas 2021, and Government of Spain's Plan Nacional PGC2018-101643-B-I00 grants. David Rossell was also partially funded by the Europa Excelencia grant EUR2020-112096 and Ramón y Cajal Fellowship RYC-2015-18544.