Training Probabilistic Machine Learning Models with Population Annealing
We consider the training of restricted Boltzmann machines (RBM) on real and synthetically generated multimodal datasets using different sampling strategies for maximum likelihood estimation (MLE). By com- paring the population annealing (PA) algorithm with the widely used persistent contrastive divergence (PCD), we demonstrate that PA leads to superior generative model training without incurring additional com- putational effort. Since PA can be efficiently implemented on modern graphics processing units (GPUs), we advocate its use as an effective method for training machine learning models having RBM components.
The biggest obstacle to using machine learning is unlabelled or poorly labelled data. Generative machine learning can help overcome this.
We describe a practical impediment to the application of deep neural network models when large training datasets are unavailable. Encouragingly however, we show that recent machine learning advances make it possible to obtain the benefits of deep neural networks by making more efficient use of training data that most practitioners do have.