RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication
Artículo científico (JournalArticle)
A cooperative digital twin multi-agent reinforcement learning for circular supply chains: balanced control across production, logistics, and sustainability
[EN] Coordination of production, inventory, logistics, and recovery decisions in circular supply chains (CSCs) remains challenging due to demand uncertainty, transport variability, and competing objectives across service, cost, and environmental dimensions. Controlled quantification of trade-offs and information value under matched experimental conditions has rarely been reported in prior digital twin (DT)-enabled control studies. To address this gap, an integrated methodological framework is proposed by coupling a simulation-based DT with cooperative multi-agent reinforcement learning (MARL). Specifically, a five-agent controller was trained to coordinate planning, inventory, logistics, expediting, and recycling decisions under a shared multi-objective reward within a DT¿MARL formulation. The primary contribution is methodological: a controlled evaluation protocol with matched seeds, fixed horizons, and 95% confidence intervals is introduced to enable reproducible comparison across baselines, disruption scenarios, and sector archetypes. Moreover, a complementary Value-of-Data (VoD) protocol was used to isolate the marginal impact of cross-functional information integration on controller performance. In benchmark experiments, balanced improvements in lead time and on-time-in-full (OTIF) delivery were observed relative to a No-Op (no-action) baseline, while policy stability was maintained under transport, demand, and energy shocks. Furthermore, transferability was demonstrated across four archetypal operating regimes without retuning. Finally, VoD analysis indicated that integrated observation regimes shifted operating points toward improved resource efficiency., This research was funded by the project titled "Gestion Integral de Materiales y Residuos en la Industria de la Construccion: Fomentando la Economia Circular mediante la Adopcion de Inteligencia Artificial y Sistemas Inteligentes " (ref. SI4/PJI/2024-00211) . The project is supported by the Comunidad de Madrid through a direct grant agreement aimed at fostering and promoting research and technology transfer at the Universidad Autonoma de Madrid . Additionally, the research leading to these results received funding from the European Union Horizon Europe Programme with grant agreement No. 101147855 Intelligent and Sustainable Building Management powered by Cross-Sectoral Lifecycle (DATAWiSE) .