Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 1
Encontrada(s) 1 página(s)

RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Q-Learning for Online PID Controller Tuning in Continuous Dynamic Systems: An Interpretable Framework for Exploring Multi-Agent Systems

  • Ibarra-Pérez, Davor Matías Samuel
  • Garcia-Nieto, Sergio|||0000-0002-2722-742X
  • Sanchís Saez, Javier|||0000-0001-9697-2696
[EN] This study proposes a discrete multi-agent Q-learning framework for online tuning of PID controllers in continuous dynamic systems with limited observability. Each PID gain (Kp, Ki, Kd) is adjusted by an independent learning agent operating in a discrete state space defined by its own gain value. At each decision step, agents choose among three actions: decrease, keep, or increase the gain, and they act simultaneously at fixed decision intervals. This design helps preserve quasi-stationary conditions from the agents’ perspective and supports convergence. Coordination is achieved through a shared cumulative global reward that combines system-performance terms with time and control-effort penalties, as well as stability incentives, guiding exploration toward control objectives. Implemented in Python, the framework is validated on two nonlinear control problems: a water-tank system and an inverted pendulum (cart-pole). The agents reach initial convergence after approximately 300 and 500 episodes, respectively, and obtain overall success rates of 49.6% and 46.2% over 5,000 training episodes. Results show sustained learning toward effective PID parameter settings capable of stabilizing both systems without requiring explicit dynamic models. These findings support the feasibility of a low-complexity, discrete reinforcement-learning approach for online adaptive PID tuning, producing interpretable and reproducible control policies and providing a foundation for future hybrid schemes combining classical control theory with reinforcement-learning agents., This research was funded by Vicerrectorado de Investigación de la Universitat Politècnica de València (PAID-11-24) and supported by Agencia Nacional de Investigación de Chile ANID BECAS/DOCTORADO EN EL EXTRANJERO 72230436-2023.