Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 5
Encontrada(s) 1 página(s)

Dipòsit Digital de Documents de la UAB
Publication Artículo científico (JournalArticle)

Impact assessment of the Strategic Planning performance in Shared U-space volumes

  • Ramos González, Juan José|||0000-0002-0881-7205
  • Liu, Zhiqiang|||0009-0006-8034-0364
  • Munoz-Gamarra, Jose Luis|||0000-0003-3213-0558
  • Pastor, Enric|||0000-0002-7587-8702
  • Barrado, Cristina|||0000-0003-0100-724X
Este trabajo analiza cómo los distintos proveedores de servicios de espacio aéreo (USSP) que gestionan un volumen de espacio aéreo compartido influyen en el rendimiento de los demás. El trabajo demuestra cómo las distintas capacidades de planificación estratégica de los USSP, que van desde los procedimientos y políticas hasta la representación de trayectorias, así como las estrategias de deconflicto, repercuten en el uso de este recurso común en términos de capacidad efectiva del espacio aéreo. El documento pone en práctica el concepto de tiempo razonable para reaccionar proponiendo un calendario de planificación con hitos comunes para la aplicación de los procedimientos de autorización requeridos antes del vuelo. Además, se compara la política de planificación por orden de llegada con una política de planificación por lotes más eficaz, en la que los lotes de planes de vuelo se procesan para mitigar los posibles conflictos existentes en la fase estratégica. Además, el artículo muestra cómo las capacidades de la USSP para representar la trayectoria y su incertidumbre asociada serán también clave para optimizar el uso del espacio aéreo. Basándose en un escenario CORUS-XUAM VLD, un análisis de simulación evaluará cómo las capacidades de los USSP para gestionar las actividades de planificación de vuelos repercuten en la ocupación efectiva del espacio aéreo, poniendo en peligro en algunos casos no sólo el rendimiento propio, sino también el de otros usuarios del espacio aéreo., This work analyses how the different U-space service providers (USSPs) managing a shared airspace volume will impact on the other's performance. The paper demonstrates how the various strategic planning USSPs capabilities, ranging from procedures and policies, trajectory representation, as well as the deconflicting strategies, impact on the use of this common resource in terms of effective airspace capacity. The paper brings to practice the concept of Reasonable Time to React by proposing a planning timeline with common milestones for the implementation of the required authorization procedures prior to the flight. The paper analyses how these milestones impact on the effective airspace capacity. Also, the First-Come First-Served planning policy is compared with respect to a more efficient batch planning policy, where flight plan batches are processed to mitigate the potential conflicts existing at the strategic phase. Furthermore, the paper discusses how the capabilities supported by the USSP to represent the trajectory and its associated uncertainty will be also key to optimize the use the airspace. Based on a CORUS-XUAM VLD scenario, a simulation analysis will assess how the USSPs' capabilities to manage flight planning activities impact on the effective occupation of airspace, jeopardizing in some cases not just the own performance but also that of other airspace users.
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Countering a drone in a 3D space: Analyzing deep reinforcement learning methods

  • Çetin, Ender|||0000-0002-9258-1919
  • Barrado Muxí, Cristina|||0000-0003-0100-724X
  • Pastor Llorens, Enric|||0000-0002-7587-8702
Unmanned aerial vehicles (UAV), also known as drones have been used for a variety of reasons and the commercial drone market growth is expected to reach remarkable levels in the near future. However, some drone users can mistakenly or intentionally fly into flight paths at major airports, flying too close to commercial aircraft or invading people’s privacy. In order to prevent these unwanted events, counter-drone technology is needed to eliminate threats from drones and hopefully they can be integrated into the skies safely. There are various counter-drone methods available in the industry. However, a counter-drone system supported by an artificial intelligence (AI) method can be an efficient way to fight against drones instead of human intervention. In this paper, a deep reinforcement learning (DRL) method has been proposed to counter a drone in a 3D space by using another drone. In a 2D space it is already shown that the deep reinforcement learning method is an effective way to counter a drone. However, countering a drone in a 3D space with another drone is a very challenging task considering the time required to train and avoid obstacles at the same time. A Deep Q-Network (DQN) algorithm with dueling network architecture and prioritized experience replay is presented to catch another drone in the environment provided by an Airsim simulator. The models have been trained and tested with different scenarios to analyze the learning progress of the drone. Experiences from previous training are also transferred before starting a new training by pre-processing the previous experiences and eliminating those considered as bad experiences. The results show that the best models are obtained with transfer learning and the drone learning progress has been increased dramatically. Additionally, an algorithm which combines imitation learning and reinforcement learning is implemented to catch the target drone. In this algorithm, called deep q-learning from demonstrations (DQfD), expert demonstrations data and self-generated data by the agent are sampled and the agent continues learning without overwriting the demonstration data. The main advantage of this algorithm is to accelerate the learning process even if there is a small amount of demonstration data., This work was funded partially by the AGAUR under grant 2020PANDE00141, the Ministry of Science and Innovation of Spain under grant PID2020-116377RB-C21 and the SESAR Joint Undertaking (JU) project CORUS-XUAM, under grant SESAR-VLD2 101017682. The JU receives support from the European Union’s Horizon 2020 research and innovation program and SESAR JU members other than the Union., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Counter a drone and the performance analysis of deep reinforcement learning method and human pilot

  • Çetin, Ender|||0000-0002-9258-1919
  • Barrado Muxí, Cristina|||0000-0003-0100-724X
  • Pastor Llorens, Enric|||0000-0002-7587-8702
Artificial Intelligence (AI) has been used in different research areas in aerospace to create an intelligent system. Especially, an unmanned aerial vehicle (UAV), known as a drone, can be controlled by AI methods such as deep reinforcement learning (DRL) in different purposes. Drones with DRL become more intelligent and eventually they can be fully autonomous. In this paper, DRL method supported by real time object detection model is proposed to detect and catch a drone. Additionally, the results are analyzed by comparing the time to catch the target drone in seconds between DRL method, human pilot and an algorithm which directs the drone towards the target position without using any AI method or navigation and guidance method. The main idea is to catch a drone in an environment as fast as possible without crashing any obstacles inside the environment. In DRL method, the agent is a quadcopter drone and it is rewarded in each time step by the environment provided by Airsim flight simulator. Drone is trained to catch the target drone by using DRL model which is based on deep Q-Network algorithm. After training, the tests have been made by the agent drone with DRL model and human pilots to catch stationary and non-stationary target drone. The training and test results show that the agent drone learns to catch target drone which can be a stationary and a non-stationary. In addition. the agent avoids crashing any obstacles in the environment with a minimum success rate of 94%. Also, DRL model performance is compared with the human pilot performances and the agent with DRL model shows better time to catch the target drone. Human pilots struggle to control the drone by using remote controller when catching the target in simulation. However, the agent with DRL model is rarely missing the target when trying to catch the target., This work was funded partially by the AGAUR under grant 2020PANDE00141, the Ministry of Science and Innovation of Spain under grant PID2020-116377RB-C21 and the SESAR Joint Undertaking (JU) project CORUS-XUAM, under grant SESAR-VLD2 101017682. The JU receives support from the European Union’s Horizon 2020 research and innovation programme and the SESAR JU members other than the Union. Special thanks to Dr. Pablo Royo Chic for his assistance to our paper by piloting a drone to catch the target drone.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Countering a drone in a 3D space: Analyzing deep reinforcement learning methods

  • Çetin, Ender
  • Barrado Muxí, Cristina
  • Pastor Llorens, Enric
Unmanned aerial vehicles (UAV), also known as drones have been used for a variety of reasons and the commercial drone market growth is expected to reach remarkable levels in the near future. However, some drone users can mistakenly or intentionally fly into flight paths at major airports, flying too close to commercial aircraft or invading people’s privacy. In order to prevent these unwanted events, counter-drone technology is needed to eliminate threats from drones and hopefully they can be integrated into the skies safely. There are various counter-drone methods available in the industry. However, a counter-drone system supported by an artificial intelligence (AI) method can be an efficient way to fight against drones instead of human intervention. In this paper, a deep reinforcement learning (DRL) method has been proposed to counter a drone in a 3D space by using another drone. In a 2D space it is already shown that the deep reinforcement learning method is an effective way to counter a drone. However, countering a drone in a 3D space with another drone is a very challenging task considering the time required to train and avoid obstacles at the same time. A Deep Q-Network (DQN) algorithm with dueling network architecture and prioritized experience replay is presented to catch another drone in the environment provided by an Airsim simulator. The models have been trained and tested with different scenarios to analyze the learning progress of the drone. Experiences from previous training are also transferred before starting a new training by pre-processing the previous experiences and eliminating those considered as bad experiences. The results show that the best models are obtained with transfer learning and the drone learning progress has been increased dramatically. Additionally, an algorithm which combines imitation learning and reinforcement learning is implemented to catch the target drone. In this algorithm, called deep q-learning from demonstrations (DQfD), expert demonstrations data and self-generated data by the agent are sampled and the agent continues learning without overwriting the demonstration data. The main advantage of this algorithm is to accelerate the learning process even if there is a small amount of demonstration data., This work was funded partially by the AGAUR under grant 2020PANDE00141, the Ministry of Science and Innovation of Spain under grant PID2020-116377RB-C21 and the SESAR Joint Undertaking (JU) project CORUS-XUAM, under grant SESAR-VLD2 101017682. The JU receives support from the European Union’s Horizon 2020 research and innovation program and SESAR JU members other than the Union., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Counter a drone and the performance analysis of deep reinforcement learning method and human pilot

  • Çetin, Ender
  • Barrado Muxí, Cristina
  • Pastor Llorens, Enric
Artificial Intelligence (AI) has been used in different research areas in aerospace to create an intelligent system. Especially, an unmanned aerial vehicle (UAV), known as a drone, can be controlled by AI methods such as deep reinforcement learning (DRL) in different purposes. Drones with DRL become more intelligent and eventually they can be fully autonomous. In this paper, DRL method supported by real time object detection model is proposed to detect and catch a drone. Additionally, the results are analyzed by comparing the time to catch the target drone in seconds between DRL method, human pilot and an algorithm which directs the drone towards the target position without using any AI method or navigation and guidance method. The main idea is to catch a drone in an environment as fast as possible without crashing any obstacles inside the environment. In DRL method, the agent is a quadcopter drone and it is rewarded in each time step by the environment provided by Airsim flight simulator. Drone is trained to catch the target drone by using DRL model which is based on deep Q-Network algorithm. After training, the tests have been made by the agent drone with DRL model and human pilots to catch stationary and non-stationary target drone. The training and test results show that the agent drone learns to catch target drone which can be a stationary and a non-stationary. In addition. the agent avoids crashing any obstacles in the environment with a minimum success rate of 94%. Also, DRL model performance is compared with the human pilot performances and the agent with DRL model shows better time to catch the target drone. Human pilots struggle to control the drone by using remote controller when catching the target in simulation. However, the agent with DRL model is rarely missing the target when trying to catch the target., This work was funded partially by the AGAUR under grant 2020PANDE00141, the Ministry of Science and Innovation of Spain under grant PID2020-116377RB-C21 and the SESAR Joint Undertaking (JU) project CORUS-XUAM, under grant SESAR-VLD2 101017682. The JU receives support from the European Union’s Horizon 2020 research and innovation programme and the SESAR JU members other than the Union. Special thanks to Dr. Pablo Royo Chic for his assistance to our paper by piloting a drone to catch the target drone., Postprint (published version)