Mostrando recursos 1 - 14 de 14

  1. Deep Learning: A Bayesian Perspective

    Polson, Nicholas G.; Sokolov, Vadim
    Deep learning is a form of machine learning for nonlinear high dimensional pattern matching and prediction. By taking a Bayesian probabilistic perspective, we provide a number of insights into more efficient algorithms for optimisation and hyper-parameter tuning. Traditional high-dimensional data reduction techniques, such as principal component analysis (PCA), partial least squares (PLS), reduced rank regression (RRR), projection pursuit regression (PPR) are all shown to be shallow learners. Their deep learning counterparts exploit multiple deep layers of data reduction which provide predictive performance gains. Stochastic gradient descent (SGD) training optimisation and Dropout (DO) regularization provide estimation and variable selection. Bayesian regularization...

  2. Uncertainty Quantification for the Horseshoe (with Discussion)

    van der Pas, Stéphanie; Szabó, Botond; van der Vaart, Aad
    We investigate the credible sets and marginal credible intervals resulting from the horseshoe prior in the sparse multivariate normal means model. We do so in an adaptive setting without assuming knowledge of the sparsity level (number of signals). We consider both the hierarchical Bayes method of putting a prior on the unknown sparsity level and the empirical Bayes method with the sparsity level estimated by maximum marginal likelihood. We show that credible balls and marginal credible intervals have good frequentist coverage and optimal size if the sparsity level of the prior is set correctly. By general theory honest confidence sets...

  3. Correction to: “Posterior Consistency of Bayesian Quantile Regression Based on the Misspecified Asymmetric Laplace Density”

    Sriram, Karthik; Ramamoorthi, R.V.
    In this note, we highlight and provide corrections to two errors in the paper: Karthik Sriram, R.V. Ramamoorthi, Pulak Ghosh (2013) “Posterior Consistency of Bayesian Quantile Regression Based on the Misspecified Asymmetric Laplace Density”, Bayesian Analysis, Vol 8, Num 2, pg 479–504.

  4. Marginal Pseudo-Likelihood Learning of Discrete Markov Network Structures

    Pensar, Johan; Nyman, Henrik; Niiranen, Juha; Corander, Jukka
    Markov networks are a popular tool for modeling multivariate distributions over a set of discrete variables. The core of the Markov network representation is an undirected graph which elegantly captures the dependence structure over the variables. Traditionally, the Bayesian approach of learning the graph structure from data has been done under the assumption of chordality since non-chordal graphs are difficult to evaluate for likelihood-based scores. Recently, there has been a surge of interest towards the use of regularized pseudo-likelihood methods as such approaches can avoid the assumption of chordality. Many of the currently available methods necessitate the use of a...

  5. Bayesian Analysis of the Stationary MAP2

    Ramírez-Cobo, P.; Lillo, R. E.; Wiper, M. P.
    In this article we describe a method for carrying out Bayesian estimation for the two-state stationary Markov arrival process ( $\mathit{MAP}_{2}$ ), which has been proposed as a versatile model in a number of contexts. The approach is illustrated on both simulated and real data sets, where the performance of the $\mathit{MAP}_{2}$ is compared against that of the well-known $\mathit{MMPP}_{2}$ . As an extension of the method, we estimate the queue length and virtual waiting time distributions of a stationary $\mathit{MAP}_{2}/G/1$ queueing system, a matrix generalization of the $M/G/1$ queue that allows for dependent inter-arrival times. Our procedure is illustrated...

  6. Asymptotic Optimality of One-Group Shrinkage Priors in Sparse High-dimensional Problems

    Ghosh, Prasenjit; Chakrabarti, Arijit
    We study asymptotic optimality of inference in a high-dimensional sparse normal means model using a broad class of one-group shrinkage priors. Assuming that the proportion of non-zero means is known, we show that the corresponding Bayes estimates asymptotically attain the minimax risk (up to a multiplicative constant) for estimation with squared error loss. The constant is shown to be 1 for the important sub-class of “horseshoe-type” priors proving exact asymptotic minimaxity property for these priors, a result hitherto unknown in the literature. An empirical Bayes version of the estimator is shown to achieve the minimax rate in case the level...

  7. The Horseshoe+ Estimator of Ultra-Sparse Signals

    Bhadra, Anindya; Datta, Jyotishka; Polson, Nicholas G.; Willard, Brandon
    We propose a new prior for ultra-sparse signal detection that we term the “horseshoe+ prior.” The horseshoe+ prior is a natural extension of the horseshoe prior that has achieved success in the estimation and detection of sparse signals and has been shown to possess a number of desirable theoretical properties while enjoying computational feasibility in high dimensions. The horseshoe+ prior builds upon these advantages. Our work proves that the horseshoe+ posterior concentrates at a rate faster than that of the horseshoe in the Kullback–Leibler (K-L) sense. We also establish theoretically that the proposed estimator has lower posterior mean squared error...

  8. Inconsistency of Bayesian Inference for Misspecified Linear Models, and a Proposal for Repairing It

    Grünwald, Peter; van Ommen, Thijs
    We empirically show that Bayesian inference can be inconsistent under misspecification in simple linear regression problems, both in a model averaging/selection and in a Bayesian ridge regression setting. We use the standard linear model, which assumes homoskedasticity, whereas the data are heteroskedastic (though, significantly, there are no outliers). As sample size increases, the posterior puts its mass on worse and worse models of ever higher dimension. This is caused by hypercompression, the phenomenon that the posterior puts its mass on distributions that have much larger KL divergence from the ground truth than their average, i.e. the Bayes predictive distribution. To...

  9. Bayesian Variable Selection Regression of Multivariate Responses for Group Data

    Liquet, B.; Mengersen, K.; Pettitt, A. N.; Sutton, M.
    We propose two multivariate extensions of the Bayesian group lasso for variable selection and estimation for data with high dimensional predictors and multi-dimensional response variables. The methods utilize spike and slab priors to yield solutions which are sparse at either a group level or both a group and individual feature level. The incorporation of group structure in a predictor matrix is a key factor in obtaining better estimators and identifying associations between multiple responses and predictors. The approach is suited to many biological studies where the response is multivariate and each predictor is embedded in some biological grouping structure such...

  10. Fast Simulation of Hyperplane-Truncated Multivariate Normal Distributions

    Cong, Yulai; Chen, Bo; Zhou, Mingyuan
    We introduce a fast and easy-to-implement simulation algorithm for a multivariate normal distribution truncated on the intersection of a set of hyperplanes, and further generalize it to efficiently simulate random variables from a multivariate normal distribution whose covariance (precision) matrix can be decomposed as a positive-definite matrix minus (plus) a low-rank symmetric matrix. Example results illustrate the correctness and efficiency of the proposed simulation algorithms.

  11. Approximate Bayesian Inference in Semiparametric Copula Models

    Grazian, Clara; Liseo, Brunero
    We describe a simple method for making inference on a functional of a multivariate distribution, based on its copula representation. We make use of an approximate Bayesian Monte Carlo algorithm, where the proposed values of the functional of interest are weighted in terms of their Bayesian exponentially tilted empirical likelihood. This method is particularly useful when the “true” likelihood function associated with the working model is too costly to evaluate or when the working model is only partially specified.

  12. Variable Selection in Seemingly Unrelated Regressions with Random Predictors

    Puelz, David; Hahn, P. Richard; Carvalho, Carlos M.
    This paper considers linear model selection when the response is vector-valued and the predictors, either all or some, are randomly observed. We propose a new approach that decouples statistical inference from the selection step in a “post-inference model summarization” strategy. We study the impact of predictor uncertainty on the model selection procedure. The method is demonstrated through an application to asset pricing.

  13. Joint Species Distribution Modeling: Dimension Reduction Using Dirichlet Processes

    Taylor-Rodríguez, Daniel; Kaufeld, Kimberly; Schliep, Erin M.; Clark, James S.; Gelfand, Alan E.
    Species distribution models are used to evaluate the variables that affect the distribution and abundance of species and to predict biodiversity. Historically, such models have been fitted to each species independently. While independent models can provide useful information regarding distribution and abundance, they ignore the fact that, after accounting for environmental covariates, residual interspecies dependence persists. With stacking of individual models, misleading behaviors, may arise. In particular, individual models often imply too many species per location. ¶ Recently developed joint species distribution models have application to presence–absence, continuous or discrete abundance, abundance with large numbers of zeros, and discrete, ordinal,...

  14. A Bayesian Nonparametric Approach to Testing for Dependence Between Random Variables

    Filippi, Sarah; Holmes, Chris C.
    Nonparametric and nonlinear measures of statistical dependence between pairs of random variables are important tools in modern data analysis. In particular the emergence of large data sets can now support the relaxation of linearity assumptions implicit in traditional association scores such as correlation. Here we describe a Bayesian nonparametric procedure that leads to a tractable, explicit and analytic quantification of the relative evidence for dependence vs independence. Our approach uses Pólya tree priors on the space of probability measures which can then be embedded within a decision theoretic test for dependence. Pólya tree priors can accommodate known uncertainty in the...

Aviso de cookies: Usamos cookies propias y de terceros para mejorar nuestros servicios, para análisis estadístico y para mostrarle publicidad. Si continua navegando consideramos que acepta su uso en los términos establecidos en la Política de cookies.