Statistics > Machine Learning
Title:Variance Reduction for Matrix Computations with Applications to Gaussian Processes
Abstract:In addition to recent developments in computing speed and memory, methodological advances have contributed to significant gains in the performance of stochastic simulation. In this paper we focus on variance reduction for matrix computations via matrix factorization. We provide insights into existing variance reduction methods for estimating the entries of large matrices. Popular methods do not exploit the reduction in variance that is possible when the matrix is factorized. We show how computing the square root factorization of the matrix can achieve in some important cases arbitrarily better stochastic performance. In addition, we detail a factorized estimator for the trace of a product of matrices and numerically demonstrate that the estimator can be up to 1,000 times more efficient on certain problems of estimating the log-likelihood of a Gaussian process. Additionally, we provide a new estimator of the log-determinant of a symmetric positive-definite matrix where the log-determinant is treated as a normalizing constant of a probability density.
| Comments: | Unable to be updated |
|---|---|
| Subjects: | Machine Learning (stat.ML) ; Machine Learning (cs.LG); Computation (stat.CO) |
| Cite as: | arXiv:2106.14565 [stat.ML] |
| (or arXiv:2106.14565v4 [stat.ML] for this version) | |
| https://doi.org/10.48550/arXiv.2106.14565 arXiv-issued DOI via DataCite |
Submission history
Access Paper:
- HTML (experimental)
Current browse context:
References & Citations
- Google Scholar
- Semantic Scholar
BibTeX formatted citation
Bookmark
Bibliographic and Citation Tools
Code, Data and Media Associated with this Article
Demos
Recommenders and Search Tools
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.