Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 40
Encontrada(s) 1 página(s)

e-cienciaDatos, Repositorio de Datos del Consorcio Madroño
Research data Set de datos (Dataset)

Art-GenEvalGPT

  • D'Haro Enríquez, Luis Fernando
  • Gil Martín, Manuel
  • Luna Jiménez, Cristina
  • Esteban Romero, Sergio
  • Estecha Garitagoitia, Marcos
  • Bellver Soler, Jaime
  • Fernández Martínez, Fernando
<p></p> <p>Description of the project</p> <p></p> <p>ASTOUND is an EIC funded project (No. 101071191) under the HORIZON-EIC-2021-PATHFINDERCHALLENGES-01 call. </p> <p></p> <p>The aim of the project is to develop an artificial conscious AI based on the Attention Schema Theory (AST) proposed by Michel Graziano. This theory proposes that consciousness arises from the brain's ability to create and maintain a simplified model of its own processing, particularly focusing attention on certain aspects of its internal and external environment. </p> <p></p> <p>The project entails creating an AI system capable of exhibiting consciousness-like behaviours by implementing principles from the AST. This involves constructing a model that simulates attentional processes, allowing the AI to prioritise and focus on relevant information while disregarding irrelevant stimuli. </p> <p></p> <p>The ASTOUND project will provide an Integrative Approach for Awareness Engineering to establish consciousness in machines, and targeting the following goals: </p> <p></p> <p>Develop an AI architecture for Artificial Consciousness based on the Attention Schema Theory (AST) through an internal model of the state of the attention. </p> <p></p> <p>Implement the proposed architecture into a contextually aware virtual agent and prove improved performance thanks to the Attention Schema; for instance, by providing coherent discussion, self-regulation, short-and-long term memory, personalisation capabilities. </p> <p></p> <p>Define novel ways to measure the presence and level of consciousness in both humans and machines.</p> <p></p> <p></p> <p>Description of the dataset</p> <p></p> <p>The dataset includes synthetic dialogues in the art domain that can be used for training a chatbot to discuss artworks within a museum setting. Leveraging Large Language Models (LLMs), particularly ChatGPT, the dataset comprises over 13,000 dialogues generated using prompt-engineering techniques. The dialogues cover a wide range of user and chatbot behaviours, including expert guidance, tutoring, and handling toxic user interactions. </p> <p></p> <p>The ArtEmis dataset serves as a basis, containing emotion attributions and explanations for artworks sourced from the WikiArt website. From this dataset, 800 artworks were selected based on consensus among human annotators regarding elicited emotions, ensuring balanced representation across different emotions. However, an imbalance in art styles distribution was noted due to the emphasis on emotional balance. </p> <p></p> <p>Each dialogue is uniquely identified using a "DIALOGUE_ID", encoding information about the artwork discussed, emotions, chatbot behaviour, and more. The dataset is structured into multiple files for efficient navigation and analysis, including metadata, prompts, dialogues, and metrics. </p> <p></p> <p>Objective evaluation of the generated dialogues was conducted, focusing on profile discrimination, anthropic behaviour detection, and toxicity evaluation. Various syntactic and semantic-based metrics are employed to assess dialogue quality, along with sentiment and subjectivity analysis. Tools like the MS Azure Content Moderator API, Detoxify library and LlamaGuard aid in toxicity evaluation. </p> <p></p> <p>The dataset's conclusion highlights the need for further work to handle biases, enhance toxicity detection, and incorporate multimodal information and contextual awareness. Future efforts will focus on expanding the dataset with additional tasks and improving chatbot capabilities for diverse scenarios.</p>
Archivo Digital UPM
Publication Contribución de congreso (ConferenceOutput)

Interpreting Sign Language Recognition Using Transformers and MediaPipe Landmarks

  • Luna Jiménez, Cristina|||0000-0001-5369-856X
  • Gil Martín, Manuel|||0000-0002-4285-6224
  • Kleinlein, Ricardo
  • San Segundo Hernández, Rubén|||0000-0001-9659-5464
  • Fernández Martínez, Fernando|||0000-0003-3877-0089
Archivo Digital UPM
Publication Artículo científico (JournalArticle)

Sign Language Dataset for Automatic Motion Generation

  • Villa Monedero, María
  • Gil Martín, Manuel|||0000-0002-4285-6224
  • Sáez Trigueros, Daniel
  • Pomirski, Andrzej
  • San Segundo Hernández, Rubén|||0000-0001-9659-5464
Archivo Digital UPM
Publication Artículo científico (JournalArticle)

Sign Language Motion Generation from Sign Characteristics

  • Gil Martín, Manuel|||0000-0002-4285-6224
  • Villa Monedero, María
  • Pomirski, Andrzej
  • Sáez Trigueros, Daniel
  • San Segundo Hernández, Rubén|||0000-0001-9659-5464
Archivo Digital UPM
Publication Contribución de congreso (ConferenceOutput)

Improving Hand Pose Recognition using Localization and Zoom Normalizations over MediaPipe Landmarks

  • Remiro Pérez, Miguel Ángel
  • Gil Martín, Manuel|||0000-0002-4285-6224
  • San Segundo Hernández, Rubén|||0000-0001-9659-5464
Archivo Digital UPM
Publication Artículo científico (JournalArticle)

Automatic Detection of Inconsistencies and Hierarchical Topic Classification for Open-Domain Chatbots

  • Rodríguez Cantelar, Mario|||0000-0001-9703-4458
  • Estecha Garitagoitia, Marcos Santiago|||0000-0001-8153-0182
  • D'Haro Enríquez, Luis Fernando|||0000-0002-3411-7384
  • Matía Espada, Fernando|||0000-0002-2198-1448
  • Córdoba Herralde, Ricardo de|||0000-0002-7136-9636
Archivo Digital UPM
Publication Contribución de congreso (ConferenceOutput)

Video Memorability Prediction From Jointly-learnt Semantic and Visual Features

  • Martín Fernández, Iván|||0009-0004-2769-9752
  • Kleinlein, Ricardo|||0000-0002-7313-7601
  • Luna Jiménez, Cristina|||0000-0001-5369-856X
  • Gil Martín, Manuel|||0000-0002-4285-6224
  • Fernández Martínez, Fernando|||0000-0003-3877-0089
Archivo Digital UPM
Publication Contribución de congreso (ConferenceOutput)

A Comprehensive Analysis of Parkinson’s Disease Detection Through Inertial Signal Processing

  • Gil Martín, Manuel|||0000-0002-4285-6224
  • Esteban Romero, Sergio|||0009-0008-6336-7877
  • Fernández Martínez, Fernando|||0000-0003-3877-0089
  • San Segundo Hernández, Rubén|||0000-0001-9659-5464
Archivo Digital UPM
Publication Contribución de congreso (ConferenceOutput)

Parkinson’s Disease Detection Through Inertial Signals and Posture Insights

  • Gil Martín, Manuel|||0000-0002-4285-6224
  • Esteban Romero, Sergio|||0009-0008-6336-7877
  • Fernández Martínez, Fernando|||0000-0003-3877-0089
  • San Segundo Hernández, Rubén|||0000-0001-9659-5464
Archivo Digital UPM
Publication Contribución de congreso (ConferenceOutput)

Overview of Robust and Multilingual Automatic Evaluation Metrics for Oopen-Domain Dialogue Systems at DSTC 11 Track 4

  • Rodríguez Cantelar, Mario|||0000-0001-9703-4458
  • Zhang, Chen
  • Tang, Chengguang
  • Shi, Ke
  • Ghazarian, Sarik
  • Sedoc, João
  • D'Haro Enríquez, Luis Fernando|||0000-0002-3411-7384
  • Rudnicky, Alexander|||0000-0003-2044-8446
Archivo Digital UPM
Publication Contribución de congreso (ConferenceOutput)

Multimodal Audio-Language Model for Speech Emotion Recognition

  • Bellver Soler, Jaime|||0009-0006-7973-4913
  • Martín Fernández, Iván|||0009-0004-2769-9752
  • Bravo Pacheco, Jose Manuel
  • Esteban Romero, Sergio|||0009-0008-6336-7877
  • Fernández Martínez, Fernando|||0000-0003-3877-0089
  • D'Haro Enríquez, Luis Fernando|||0000-0002-3411-7384
Archivo Digital UPM
Publication Contribución de congreso (ConferenceOutput)

THAURUS: An Innovative Multimodal Chatbot Based on the Next Generation of Conversational AI

  • Estecha Garitagoitia, Marcos Santiago|||0000-0001-8153-0182
  • Rodríguez Cantelar, Mario|||0000-0001-9703-4458
  • Garrachón Ruiz, Alfredo
  • Fernández García, Claudia Garoé
  • Esteban Romero, Sergio|||0009-0008-6336-7877
  • Conforto López, Cristina
  • Saiz Fernández, Alberto
  • Fernández Salvador, Luis Fernando
  • D'Haro Enríquez, Luis Fernando|||0000-0002-3411-7384
Archivo Digital UPM
Publication Artículo científico (JournalArticle)

Evaluating emotional and subjective responses in synthetic art-related dialogues: A multi-stage framework with large language models

  • Luna Jiménez, Cristina|||0000-0001-5369-856X
  • Gil Martín, Manuel|||0000-0002-4285-6224
  • D'Haro Enríquez, Luis Fernando|||0000-0002-3411-7384
  • Fernández Martínez, Fernando|||0000-0003-3877-0089
  • San Segundo Hernández, Rubén|||0000-0001-9659-5464
Archivo Digital UPM
Publication Contribución de congreso (ConferenceOutput)

Ethical Foresight of Artificial Consciousness: The Case for a “Conscious Copilot”

  • Salgado Criado, Jesús|||0000-0002-4513-6273
  • Elamrani, Aida|||0000-0002-6873-8795
  • Fernández Aller, María Celia|||0000-0002-0642-2058
Archivo Digital UPM
Publication Artículo científico (JournalArticle)

A dataset of synthetic art dialogues with ChatGPT

  • Gil Martín, Manuel|||0000-0002-4285-6224
  • Luna Jiménez, Cristina|||0000-0001-5369-856X
  • Esteban Romero, Sergio|||0009-0008-6336-7877
  • Estecha Garitagoitia, Marcos Santiago|||0000-0001-8153-0182
  • Fernández Martínez, Fernando|||0000-0003-3877-0089
  • D'Haro Enríquez, Luis Fernando|||0000-0002-3411-7384
Archivo Digital UPM
Publication Artículo científico (JournalArticle)

Dual Leap Motion Controller 2: A Robust Dataset for Multi-view Hand Pose Recognition

  • Gil Martín, Manuel|||0000-0002-4285-6224
  • Marini, Marco
  • San Segundo Hernández, Rubén|||0000-0001-9659-5464
  • Cinque, Luigi
Archivo Digital UPM
Publication Contribución de congreso (ConferenceOutput)

Larger encoders, smaller regressors: exploring label dimensionality reduction and multimodal large language models as feature extractors for predicting social perception

  • Martín Fernández, Iván|||0009-0004-2769-9752
  • Esteban Romero, Sergio|||0009-0008-6336-7877
  • Bellver Soler, Jaime|||0009-0006-7973-4913
  • Fernández Martínez, Fernando|||0000-0003-3877-0089
  • Gil Martín, Manuel|||0000-0002-4285-6224
Archivo Digital UPM
Publication Contribución de congreso (ConferenceOutput)

Multilingual Speech Emotion Recognition combining Audio and Text Information using Small Language Models

  • Bellver Soler, Jaime|||0009-0006-7973-4913
  • Rodríguez Cantelar, Mario|||0000-0001-9703-4458
  • Estecha Garitagoitia, Marcos Santiago|||0000-0001-8153-0182
  • Córdoba Herralde, Ricardo de|||0000-0002-7136-9636
  • D'Haro Enríquez, Luis Fernando|||0000-0002-3411-7384
Archivo Digital UPM
Publication Artículo científico (JournalArticle)

Frequency analysis and transfer learning across different body sensor locations in parkinson’s disease detection using inertial signals

  • Rey Díaz, Alejandro
  • Martín Fernández, Iván|||0009-0004-2769-9752
  • San Segundo Hernández, Rubén|||0000-0001-9659-5464
  • Gil Martín, Manuel|||0000-0002-4285-6224
Archivo Digital UPM
Publication Artículo científico (JournalArticle)

Transformer-based prediction of hospital readmissions for diabetes patients

  • García Mosquera, Jorge|||0009-0002-6840-6544
  • Villa Monedero, María
  • Gil Martí­n, Manuel|||0000-0002-4285-6224
  • San Segundo Hernández, Rubén|||0000-0001-9659-5464
Archivo Digital UPM
Publication Artículo científico (JournalArticle)

Overview of the ninth dialog system technology challenge: DSTC9

  • Gunasekara, Chulaka|||0000-0001-9272-3684
  • Kim, Seokhwan|||0000-0002-7443-1212
  • D'Haro Enríquez, Luis Fernando|||0000-0002-3411-7384
  • Rastogi, Abhinav|||0009-0008-0627-3153
  • Chen, Yun-Nung|||0000-0003-1777-3942
  • Eric, Mihail
  • Hedayatnia, Behnam
  • Gopalakrishnan, Karthik
  • Liu, Yang
  • Huang, Chao-Wei
  • Hakkani Tur, Dilek|||0000-0001-5246-2117
  • Li, Jinchao
  • Zhu, Qi
  • Luo, Lingxiao
  • Liden, Lars
  • Huang, Kaili|||0009-0007-5393-3479
  • Shayandeh, Shahin
  • Liang, Runze
  • Peng, Baolin
  • Zhang, Zheng
  • Shukla, Swadheen
  • Huang, Minlie|||0000-0001-7111-1849
  • Gao, Jianfeng
  • Mehri, Shikib
  • Feng, Yulan
  • Gordon, Carla
  • Alavi, Seyed Hossein
  • Traum, David|||0000-0003-3473-9586
  • Eskenazi, Maxine|||0009-0009-6357-104X
  • Beirami, Ahmad
  • Cho, Eunjoon
  • Crook, Paul A.|||0000-0002-0252-0769
  • De, Ankita
  • Geramifard, Alborz
  • Satwik, Kottur
  • Moon, Seungwhan|||0009-0003-2507-1884
  • Poddar, Shivani
  • Subba, Rajen
Archivo Digital UPM
Publication Contribución de congreso (ConferenceOutput)

Cutting Through Overload: Efficient Token Dropping for Speech Emotion Recognition in Multimodal Large Language Models

  • Bellver Soler, Jaime|||0009-0006-7973-4913
  • Rodríguez Cantelar, Mario|||0000-0001-9703-4458
  • Córdoba Herralde, Ricardo de|||0000-0002-7136-9636
  • D'Haro Enríquez, Luis Fernando|||0000-0002-3411-7384
Archivo Digital UPM
Publication Contribución de congreso (ConferenceOutput)

Design, Generation and Evaluation of a Synthetic Dialogue Dataset for Contextually Aware Chatbots in Art Museums

  • Rachidi, Inass
  • Ezzakri, Anas
  • Bellver Soler, Jaime|||0009-0006-7973-4913
  • D'Haro Enríquez, Luis Fernando|||0000-0002-3411-7384
Archivo Digital UPM
Publication Contribución de congreso (ConferenceOutput)

Context or Retrieval? Evaluating RAG Methods for Art and Museum QA System

  • Ramos Varela, Samuel
  • Bellver Soler, Jaime|||0009-0006-7973-4913
  • Estecha Garitagoitia, Marcos Santiago|||0000-0001-8153-0182
  • D'Haro Enríquez, Luis Fernando|||0000-0002-3411-7384
Archivo Digital UPM
Publication Contribución de congreso (ConferenceOutput)

STEPI: System for Triplet Extraction of Personal Information in dialogue systems

  • Villa Monedero, María
  • Bellver Soler, Jaime|||0009-0006-7973-4913
  • Ghebriout, Mohamed Imed Eddine|||0009-0006-6938-7110
  • Córdoba Herralde, Ricardo de|||0000-0002-7136-9636
  • D'Haro Enríquez, Luis Fernando|||0000-0002-3411-7384
Archivo Digital UPM
Publication Artículo científico (JournalArticle)

Synthesizing olfactory understanding: multimodal language models for image-text smell matching

  • Esteban Romero, Sergio|||0009-0008-6336-7877
  • Martín Fernández, Iván|||0009-0004-2769-9752
  • Gil Martín, Manuel|||0000-0002-4285-6224
  • Fernández Martínez, Fernando|||0000-0003-3877-0089
Archivo Digital UPM
Publication Contribución de congreso (ConferenceOutput)

LLM-driven multimodal video-text fusion for isolated sign language recognition

  • Esteban Romero, Sergio|||0009-0008-6336-7877
  • Luna Jiménez, Cristina|||0000-0001-5369-856X
  • Gil Martín, Manuel|||0000-0002-4285-6224
  • Fernández Martínez, Fernando|||0000-0003-3877-0089
  • Andre, Elizabeth
Archivo Digital UPM
Publication Tesis doctoral (DoctoralThesis)

Personal Assistant with Emotional and Multilingual Capabilities for Social Robots

Archivo Digital UPM
Publication Artículo científico (JournalArticle)

Isolated German sign language recognition for classifying polar answers using landmarks and lightweight transformers

  • Luna Jiménez, Cristina|||0000-0001-5369-856X
  • Eing, Lenart|||0009-0005-8855-7659
  • Esteban Romero, Sergio|||0009-0008-6336-7877
  • Gil Martin, Manuel|||0000-0002-4285-6224
  • Andre, Elizabeth|||0000-0002-2367-162X
Archivo Digital UPM
Publication Contribución de congreso (ConferenceOutput)

ASSIST: A Multi-Agentic Framework for Human Computer Interaction in Cultural Heritage settings

  • Ramos Varela, Samuel|||0009-0000-8458-6202
  • Guragain, Anmol|||0009-0009-8491-8663
  • Bellver Soler, Jaime|||0009-0006-7973-4913
  • Aragón Diaz, David
  • Long, Lin
  • D'Haro Enríquez, Luis Fernando|||0000-0002-3411-7384
Archivo Digital UPM
Publication Artículo científico (JournalArticle)

Principled evaluation of multi-label persuasion in advertisements with large vision-language models

  • Martín Fernández, Iván|||0009-0004-2769-9752
  • Constantin, Mihai Gabriel|||0000-0002-2312-6672
  • Ionescu, Bogdan
  • Gil Martín, Manuel|||0000-0002-4285-6224
  • Fernández Martínez, Fernando|||0000-0003-3877-0089
Archivo Digital UPM
Publication Artículo científico (JournalArticle)

A comprehensive study on contrastive pre-training and fine tuning of vision and text transformers for video memorability prediction

  • Martín Fernández, Iván|||0009-0004-2769-9752
  • Esteban Romero, Sergio|||0009-0008-6336-7877
  • Gil Martín, Manuel|||0000-0002-4285-6224
  • Fernández Martínez, Fernando|||0000-0003-3877-0089
Archivo Digital UPM
Publication Artículo científico (JournalArticle)

Real-world road damage dataset with potholes, cracks, and maintenance holes

  • Giordani, Enrico
  • Arcioni, Lorenzo|||0009-0001-0963-3993
  • Gil Martín, Manuel|||0000-0002-4285-6224
  • Foresti, Gian Luca|||0000-0002-8425-6892
  • Marini, Marco Raoul|||0000-0002-2540-2570
Archivo Digital UPM
Publication Contribución de congreso (ConferenceOutput)

A Personalized, Multimodal AI Assistant for Enhancing Museum Visitor Experience

  • Guragain, Anmol|||0009-0009-8491-8663
  • Bellver Soler, Jaime|||0009-0006-7973-4913
  • Ramos Varela, Samuel|||0009-0000-8458-6202
  • Lin, Long
  • Aragón Diaz, David
  • D'Haro Enríquez, Luis Fernando|||0000-0002-3411-7384
Archivo Digital UPM
Publication Contribución de congreso (ConferenceOutput)

Commonsense Generation and Evaluation for Dialogue Systems using Large Language Models

  • Estecha Garitagoitia, Marcos Santiago|||0000-0001-8153-0182
  • Zhang, Chen
  • Rodríguez Cantelar, Mario|||0000-0001-9703-4458
  • D'Haro Enríquez, Luis Fernando|||0000-0002-3411-7384
Archivo Digital UPM
Publication Contribución de congreso (ConferenceOutput)

DREAM: A Multicultural Multimodal Dataset Linking Dialogues and Realistic Image Sequences

  • Mallo de la Calle, Juan
  • Estecha Garitagoitia, Marcos Santiago|||0000-0001-8153-0182
  • Córdoba Herralde, Ricardo de|||0000-0002-7136-9636
  • D'Haro Enríquez, Luis Fernando|||0000-0002-3411-7384
Archivo Digital UPM
Publication Contribución de congreso (ConferenceOutput)

Can large sound event detection models be accurately adapted to specific acoustic scenarios?

  • Fernández Castañón, Ramón
  • Espinoza Cuadros, Fernando Manuel|||0000-0001-8051-0884
  • Perero Codosero, Juan Manuel
  • Sancho Lozano, Ester
  • Hernández Gómez, Luis Alfonso|||0000-0003-1481-9087
Archivo Digital UPM
Publication Contribución de congreso (ConferenceOutput)

Exploring Video Transformers and Automatic Segment Selection for Memorability Prediction

  • Martín Fernández, Iván|||0009-0004-2769-9752
  • Esteban Romero, Sergio|||0009-0008-6336-7877
  • Bellver Soler, Jaime|||0009-0006-7973-4913
  • Gil Martín, Manuel|||0000-0002-4285-6224
  • Fernández Martínez, Fernando|||0000-0003-3877-0089
Archivo Digital UPM
Publication Contribución de congreso (ConferenceOutput)

THAU-UPM at EmoSPeech-IberLEF2024: Efficient Adaptation of Mono-modal and Multi-modal Large Language Models for Automatic Speech Emotion Recognition

  • Esteban Romero, Sergio|||0009-0008-6336-7877
  • Bellver Soler, Jaime|||0009-0006-7973-4913
  • Martín Fernández, Iván|||0009-0004-2769-9752
  • Gil Martín, Manuel|||0000-0002-4285-6224
  • D'Haro Enríquez, Luis Fernando|||0000-0002-3411-7384
  • Fernández Martínez, Fernando|||0000-0003-3877-0089