Pasar al contenido principal

ESFUERZO CONJUNTO ENTRE BIOLOGIA Y TECNOLOGIA PARA MONITOREAR Y RECUPERAR ESPECIES Y ECOSISTEMAS IMPACTADOS POR LA PESCA: CONECTIVIDAD ESPACIAL E INDICADORES ECOLOGICOS

PID2020-114732RB-C31

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal de I+D+i Orientado a los Retos de la Sociedad
Convocatoria Proyectos de I+D+i (Generación de Conocimiento y Retos Investigación)
Año convocatoria 2020
Plan Estatal 2017-2020 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario AGENCIA ESTATAL CONSEJO SUPERIOR DE INVESTIGACIONES CIENTIFICAS (CSIC)

Resultados relacionados

Resultados totales (Incluyendo duplicados): 3
Encontrada(s) 1 página(s)

DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Evaluating the effectiveness of a deep-sea regional MPA network for Nephrops norvegicus using connectivity indicators

  • Clavel-Henry, Morane
  • López-Aguilar, Miguel
  • Bahamon, Nixon
  • Aguzzi, Jacopo
  • Navarro, Joan
  • Company, Joan B.
2nd Marine Socio-Ecological Systems Symposium (MSEAS), 3-7 June 2024, Yokohama, Japan.-- 1 page, Information on marine connectivity, as the link between populations through the dispersal of individuals, is a valuable resource for policy makers involved in spatial conservation planning decisions. One of the objectives of marine governance and the 2030 horizon is to assess the effectiveness of marine protected areas (MPAs). A large network of 35 MPAs has recently been implemented in the northwestern Mediterranean Sea and a programme of monitoring and assessment has been initiated with the support of stakeholders and research projects. However, robust information on the potential connectivity within and beyond these MPAs is lacking. We propose new indicators of connectivity that allow to compare the establishment and temporal variability of connections in MPAs. The indicators are based on larval transport simulations, which are the results of modelling tools combining ecology, physics, and technology and represent three aspects of the connectivity: occurrence, quantity, and frequency. These indicators have been tested on MPAs designed to restore and protect a commercially important species, Nephrops norvegicus. We expect these indicators to highlight the persistence and variability of connectivity in each MPA throughout the species’ spawning season (January to March). The connectivity of the entire MPA network is likely to be limited due to the presence of mesoscale circulations along a density front. However, connectivity within sub-groups of MPAs distributed on different sides of the front is assumed to persist throughout the spawning season. Our study is a preliminary analysis to support regional marine governance decisions that may be applicable elsewhere, PLOME (PLEC2021-007525/AEI/10.13039/501100011033 ); BITER (PID2020-114732RB-C31), Peer reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Simplifying deep temporal difference learning

  • Gallici, Matteo|||0000-0002-4898-2766
  • Fellows, Mattie
  • Ellis, Benjamin
  • Pou Mulet, Bartomeu
  • Masmitjà Rusiñol, Ivan|||0000-0001-6355-7955
  • Foerster, Jakob Nicolaus
  • Martín Muñoz, Mario|||0000-0002-4125-6630
Q-learning played a foundational role in the field reinforcement learning (RL). However, TD algorithms with off-policy data, such as -learning, or nonlinear function approximation like deep neural networks require several additional tricks to stabilise training, primarily a large replay buffer and target networks. Unfortunately, the delayed updating of frozen network parameters in the target network harms the sample efficiency and, similarly, the large replay buffer introduces memory and implementation overheads. In this paper, we investigate whether it is possible to accelerate and simplify off-policy TD training while maintaining its stability. Our key theoretical result demonstrates for the first time that regularisation techniques such as LayerNorm can yield provably convergent TD algorithms without the need for a target network or replay buffer, even with off-policy data. Empirically, we find that online, parallelised sampling enabled by vectorised environments stabilises training without the need for a large replay buffer. Motivated by these findings, we propose PQN, our simplified deep online -Learning algorithm. Surprisingly, this simple algorithm is competitive with more complex methods like: Rainbow in Atari, PPO-RNN in Craftax, QMix in Smax, and can be up to 50x faster than traditional DQN without sacrificing sample efficiency. In an era where PPO has become the go-to RL algorithm, PQN reestablishes off-policy -learning as a viable alternative., Mattie Fellows is funded by a generous grant from the UKRI Engineering and Physical Sciences Research Council EP/Y028481/1. Jakob Nicolaus Foerster is partially funded by the UKRI grant EP/Y028481/1 (originally selected for funding by the ERC). Jakob Nicolaus Foerster is also supported by the JPMC Research Award and the Amazon Research Award. Matteo Gallici was partially founded by the FPI-UPC Santander Scholarship FPI-UPC_93. Ivan Masmitja is partially founded by the European Union’s Horizon Europe programme under grant agreement No 101112883, as part of DIGI4ECO. This work also acknowledges the Spanish Ministerio de Ciencia, Innovacion y Universidades (BITER-ECO: PID2020- 114732RBC31), the the Spanish National Program Ramon y Cajal RYC2022-038056-I (IM) and "Severo Ochoa Centre of Excellence" accreditation (CEX2019-000928-S)., Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Simplifying deep temporal difference learning

  • Gallici, Matteo
  • Fellows, Mattie
  • Ellis, Benjamin
  • Pou Mulet, Bartomeu
  • Masmitjà Rusiñol, Ivan
  • Foerster, Jakob Nicolaus
  • Martín Muñoz, Mario
Q-learning played a foundational role in the field reinforcement learning (RL). However, TD algorithms with off-policy data, such as -learning, or nonlinear function approximation like deep neural networks require several additional tricks to stabilise training, primarily a large replay buffer and target networks. Unfortunately, the delayed updating of frozen network parameters in the target network harms the sample efficiency and, similarly, the large replay buffer introduces memory and implementation overheads. In this paper, we investigate whether it is possible to accelerate and simplify off-policy TD training while maintaining its stability. Our key theoretical result demonstrates for the first time that regularisation techniques such as LayerNorm can yield provably convergent TD algorithms without the need for a target network or replay buffer, even with off-policy data. Empirically, we find that online, parallelised sampling enabled by vectorised environments stabilises training without the need for a large replay buffer. Motivated by these findings, we propose PQN, our simplified deep online -Learning algorithm. Surprisingly, this simple algorithm is competitive with more complex methods like: Rainbow in Atari, PPO-RNN in Craftax, QMix in Smax, and can be up to 50x faster than traditional DQN without sacrificing sample efficiency. In an era where PPO has become the go-to RL algorithm, PQN reestablishes off-policy -learning as a viable alternative., Mattie Fellows is funded by a generous grant from the UKRI Engineering and Physical Sciences Research Council EP/Y028481/1. Jakob Nicolaus Foerster is partially funded by the UKRI grant EP/Y028481/1 (originally selected for funding by the ERC). Jakob Nicolaus Foerster is also supported by the JPMC Research Award and the Amazon Research Award. Matteo Gallici was partially founded by the FPI-UPC Santander Scholarship FPI-UPC_93. Ivan Masmitja is partially founded by the European Union’s Horizon Europe programme under grant agreement No 101112883, as part of DIGI4ECO. This work also acknowledges the Spanish Ministerio de Ciencia, Innovacion y Universidades (BITER-ECO: PID2020- 114732RBC31), the the Spanish National Program Ramon y Cajal RYC2022-038056-I (IM) and "Severo Ochoa Centre of Excellence" accreditation (CEX2019-000928-S)., Peer Reviewed, Postprint (published version)