Shaving weights with Occam's razor: Bayesian sparsification for neural networks using the marginal likelihood

Neural network sparsification is a promising avenue to save computational time and memory costs, especially in an age where many successful AI models are becoming too large to naively deploy on consumer hardware. While much work has focused on different weight pruning criteria, the overall sparsifiability of the network, ie, its capacity to be pruned without quality loss, has often been overlooked.