RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication
Artículo científico (JournalArticle)
Efficient training of energy-based models via spin-glass control
[EN] We introduce a new family of energy-based probabilistic graphical models for efficient unsupervised learning. Its definition is motivated by the control of the spin-glass properties of the Ising model described by the weights of Boltzmann machines. We use it to learn the Bars and Stripes dataset of various sizes and the MNIST dataset, and show how they quickly achieve the performance offered by standard methods for unsupervised learning. Our results indicate that the standard initialization of Boltzmann machines with random weights equivalent to spin-glass models is an unnecessary bottleneck in the process of training. Furthermore, this new family allows for very easy access to low-energy configurations, which points to new, efficient training algorithms. The simplest variant of such algorithms approximates the negative phase of the log-likelihood gradient with no Markov chain Monte Carlo sampling costs at all, and with an accuracy sufficient to achieve good learning and generalization., ML and AA groups acknowledge the Spanish Ministry MINECO and State Research Agency AEI (FIDEUA PID2019-106901GBI00/10.13039/501100011033, Severo Ochoa Grant Nos. SEV-2015-0522 and CEX2019-000910-S, FPI), the European Social Fund, Fundacio Cellex, Fundacio Mir-Puig, Generalitat de Catalunya (AGAUR Grant Nos. 2017 SGR 1341 and SGR 1381, CERCA program, QuantumCAT U16-011424, co-funded by ERDF Operational Program of Catalonia 2014-2020), ERC AdG NOQIA and CERQUTE, EU FEDER, MINECO-EU QUANTERA MAQS (funded by the State Research Agency AEI PCI2019-111828-2/10.13039/501100011033), the National Science Centre, Poland-Symfonia Grant No. 2016/20/W/ST4/00314 and the AXA Chair in Quantum Information Science. A P-K acknowledges funding from Fundacio Obra Socialla Caixa' (LCF/BQ/ES15/10360001) and the European Union's Horizon 2020 research and innovation programme-Grant Agreement No. 648913. G M-G acknowledges funding from Fundacio Obra Social 'la Caixa' (LCF-ICFO grant). M A G-M acknowledges funding from the Spanish Ministry of Education and Vocational Training (MEFP) through the Beatriz Galindo program 2018 (BEAGAL18/00203).
Project:
Agencia Estatal de Investigación / Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020 PCI2019-111828-2
,
European Commission / H2020 648913
,
Generalitat de Catalunya 2017 SGR 1341
,
Generalitat de Catalunya SGR 1381
,
Generalitat Valenciana QuantumCAT _U16-011424
,
National Science Centre 2016/20/W/ST4/00314
,
Polonia LCF/BQ/ES15/10360001