UPCommons. Portal del coneixement obert de la UPC
Publication
Artículo científico (JournalArticle)
Low-cost training of image-to-image diffusion models with incremental learning and task/domain adaptation
Diffusion models specialized in image-to-image translation tasks, like inpainting and colorization, have outperformed the state of the art, yet their computational requirements are exceptionally demanding. This study analyzes different strategies to train image-to-image diffusion models in a low-resource setting. The studied strategies include incremental learning and task/domain transfer learning. First, a base model for human face inpainting is trained from scratch with an incremental learning strategy. The resulting model achieves an FID score almost equivalent to that of its batch learning equivalent while significantly reducing the training time. Second, the base model is fine-tuned to perform a different task, image colorization, and, in a different domain, landscape images. The resulting colorization models showcase exceptional performances with a minimal number of training epochs. We examine the impact of different configurations and provide insights into the ability of image-to-image diffusion models for transfer learning across tasks and domains., This work is partially supported by the Spanish Ministry of Science and Innovation under contracts PID2019-107255GB and PID2021-124463OB-IOO, by the Generalitat de Catalunya under grants 2017-SGR-962, 2021-SGR-00326, 2021-SGR-00478 and DRAC (IU16-011591). The research leading to these results has received funding from the European Union’s Horizon 2020 research and innovation program under the HORIZON-AG PHOENI2X (101070586) and HORIZON-EU VITAMIN-V (101093062) projects., Peer Reviewed
Project:
Agencia Estatal de Investigación / Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020 PID2019-107255GB-C22
,
Agencia Estatal de Investigación / Plan Estatal de Investigación Científica y Técnica y de Innovación 2021-2023 PID2021-124463OB-I00
,
European Commission / HE 101070586
,
European Commission / HE 101093062