Skip to main content

RESET-UPF: REFORMULANDO ECOSISTEMAS ESCALABLES EDUCATIVOS

TIN2014-53199-C3-3-R

•
Funding agency name Ministerio de Economía y Competitividad
Funding agency acronym MINECO
Program Programa Estatal de I+D+i Orientado a los Retos de la Sociedad
Call Proyectos I+D+i Retos de Investigación
Call year 2014
State Plan 2013-2016 Plan Estatal de Investigación Científica y Técnica y de Innovación
Management unit Secretaría de Estado de Educación, Formación Profesional y Universidades
Beneficiary centre UNIVERSITAT POMPEU FABRA CCT

Resultados relacionados

Resultados totales (Incluyendo duplicados): 46
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

When we talk about older people in HCI, who are we talking about? Towards a ‘turn to community’ in the design of technologies for a growing ageing population

  • Righi, Valeria
  • Sayago Barrantes, Sergio
  • Blat, Josep
This paper addresses a number of challenges HCI designers and researchers deal with when designing digital technologies for older people. We conducted a 5-year research-through-design study, which combined ethnography, participatory design and real-life evaluation, to explore the design and use of technologies aimed to enhance the social life of older people. The paper explores widespread assumptions about the social category of older people and the meaning of technologies for them. This paper argues that assuming that (i) older people have unique needs and interests that set them apart from other user groups and (ii) the meaning of technologies remains unaltered on completion of co-design activities are both problematic, because they do not consider carefully enough the sociocultural contexts in which older people interact and engage in their daily lives. The paper posits that the meaning of technologies for older people and their own identities as individuals of a certain age are shaped within situated communities. Thus, technologies designed ‘for older people’ should be designed to meet situated and dynamic needs/interests of the communities (and not only of care) to which they belong. We draw upon our findings and other situated epistemological discourses in HCI to introduce a different perspective, a turn to community, in the design of technologies for an ever-increasing ageing population., This research has been partially funded by the Spanish Ministry of Economy and Competitiveness (RESET TIN2014-53199-C3-3-R), the EU through Life2.0 (CIP ICT PSP-2009-4-270965), and Fundación General CSIC and Obra Social ‘‘la Caixa’’ through the WorthPlay project.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

An integrated environment for learning design

  • Hernández Leo, Davinia
  • Asensio-Pérez, Juan I.
  • Derntl, Michael
  • Pozzi, Francesca
  • Chacón Pérez, Jonathan, 1986-
  • Prieto, Luis P.
  • Persico, Donatella
Learning design tools aim at supporting practitioners in their task of creating more innovative and effective computer-supported learning situations. Despite there being a myriad of proposed tools, their use presents challenges that recent studies link with practitioners' varied pedagogical approaches and context restrictions, as well as with barriers to practical application derived from the fact that most tools only cover limited functionality and do not support cooperation between practitioners. In this paper we investigate whether it is possible to provide a flexible community system that supports multiple learning design tasks. We propose an Integrated Learning Design Environment (ILDE), which is a networked system integrating collaboration functions, design editors and middleware that enables deployment of the designed learning situations into Virtual Learning Environments. We describe the iterative user-centered process adopted in the design of ILDE as well as its architecture. The architecture is implemented to show its feasibility and that it is capable of providing the targeted functionality. We also present the results of its use in training workshops with 148 practitioners from five different institutions in vocational training, higher and adult education. Some of the learning designs were deployed in VLEs and enacted with students in real learning situations., This work has been partially funded by the EC, EACEA, METIS Project 531262-LLP-2012-ES-KA3-KA3MP, with additional support by TIN2014-53199-C3-3-R, TIN2017-85179-C3-3-R, MDM-2015-0502, RecerCaixa CoT, TIN2014-53199-C3-2-R, TIN2017-85179-C3-2-R, VA082U16 and a Marie Curie Fellowship (MIOCTI, FP7-PEOPLE-2012-IEF project no. 327384).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

EdCrumble, a data-enriched visual authoring design tool for blended learning

  • Albó, Laia
  • Hernández Leo, Davinia
This paper presents an evaluation of edCrumble, a blended learning authoring tool for teachers. The tool visually represents learning designs and integrates data analytics to scaffold teacher design decisions. In addition to assessing the usability of edCrumble using UMUX questionnaire, analyses of participant views on the advantages and disadvantages of the tool in comparison to traditional course planning approaches and participant-elicited factors that can facilitate or hinder adoption of the tool are presented. Three evaluation workshops were held with teachers of different backgrounds, educational stakeholders, and students. Data-gathering instruments included interviews and questionnaires to collect qualitative and quantitative data. Further, design artifacts resulting from the workshops were analysed. The evaluation shows that the tool possesses specific features that facilitate the representation of and support for designing blended learning, uncovers the factors that may promote or inhibit its adoption, exposes its connection with solving actual educational challenges, and reveals the strengths and weaknesses in the tools usability., This work was supported by the Spanish Ministry of Science (TIN2014-53199-C3-3-R and TIN2017-85179-C3-3-R), the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502), RecerCaixa (CoT project) and the UPF PlaQUID2012. D.H.L acknowledges the support by ICREA under the ICREA Academia programme. (Corresponding author: Laia Albó).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Supporting collaborative design activity in a multi-user digital design ecology

  • Martinez-Maldonado, Roberto
  • Goodyear, Peter
  • Carvalho, Lucila
  • Thompson, Kate
  • Hernández Leo, Davinia
  • Dimitriadis, Yannis
  • Prieto, Luis P.
  • Wardak, Dewa
Across a broad range of design professions, there has been extensive research on design practices and considerable progress in creating new computer-based systems that support design work. Our research is focused on educational/instructional design for students' learning. In this sub-field, progress has been more limited. In particular, neither research nor systems development have paid much attention to the fact that design is becoming a more collaborative endeavor. This paper reports the latest research outcomes from R&D in the Educational Design Studio (EDS), a facility developed iteratively over four years to support and understand collaborative, real-time, co-present design work. The EDS serves to (i) enhance our scientific understanding of design processes and design cognition and (ii) provide insights into how designers' work can be improved through appropriate technological support. In the study presented here, we introduced a complex, multi-user, digital design tool into the existing ecology of tools and resources available in the EDS. We analysed the activity of four pairs of ‘teacher-designers’ during a design task. We identified different behaviors - in reconfiguring the task, the working methods and toolset usage. Our data provide new insights about the affordances of different digital and analogue design surfaces used in the Studio., This work was funded by the Australian Research Council (Laureate Fellowship Grant FL100100203) and by the Spanish Ministry (TIN2014-53199-C3-3-R, MDM-2015-0502). Studies presented in this paper were conducted under human ethics protocol number 2012/2794, entitled “Learning, technology and design: architectures for productive networked learning - Using complementary methods of analysis".
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Co-designing with a community of older learners for over 10 years by moving user-driven participation from the margin to the centre

  • Righi, Valeria
  • Sayago Barrantes, Sergio
  • Rosales, Andrea
  • Ferreira, Susan M.
  • Blat, Josep
This paper addresses a gap in the Participatory Design (PD) literature, wherein more research on the long-term impact of design projects is warranted. This paper reflects on a 10-year study that intertwined ethnography and 2 PD projects in a community of older learners. Although the goal of our study was to design new digital technologies, the process of designing them presented us with opportunities that gave rise to new non-digital practices, which turned out to be the legacy and most significant outcomes of the PD projects. This result invited us to review the trajectory that led to these outcomes. Our analysis shows that the most important legacy aspect of the projects arose from unexpected forms of user—driven participation that we allowed to co-exist together with those practices more related to the design goals of the PD projects. Drawing upon our results, this paper posits that engagement with PD participants that unfolds over an extended period of time is instrumental in facilitating the development of participation, understanding more deeply long-lasting project outcomes, and legitimising forms of participation that are not directly related to project/design goals., We are indebted to all the participants who took part in our study for having allowed us to become members and feel part of their communities. This research has been partially funded by the Spanish Ministry of Economy and Competitiveness (RESET TIN2014-53199-C3- 3-R ), the EU (CIP ICT PSP-2009-4-270965 ), and Fundación General CSIC and Obra Social ‘‘la Caixa’’. The authors also acknowledge the suggestions made by the reviewers.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

An actionable orchestration dashboard to enhance collaboration in the classroom

  • Amarasinghe, Ishari
  • Hernández Leo, Davinia
  • Michos, Konstantinos
  • Vujovic, Milica
The orchestration of collaborative learning activities in technology-enhanced classrooms has become a nontrivial endeavor for educators. Depending on the behaviors and needs of students that emerge in real educational situations, educators may need to orchestrate activity adaptations on the fly. These adaptations may range from the provision of additional scaffolding by the educator (e.g., the educator's participation in a group discussion) to a change in the planned pedagogical scenario (e.g., the duration). This article aims to contribute to the orchestration of technology-mediated collaborative learning sessions in a classroom context. We present the design, implementation, and evaluation of a teacher-facing dashboard that supports teachers in orchestrating scripted collaboration. Evaluation studies were conducted in 16 classroom sessions. The findings indicate that teachers found the information on the dashboard to be actionable and help facilitate just in time support to student groups., This work has been partially funded by "la Caixa Foundation" (CoT project, 100010434) and the National Research Agency of the Spanish Ministry of Science, Innovations and Universities MDM-2015- 0502, TIN2014-53199-C3-3-R, TIN2017-85179-C3-3-R.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Smartphones or laptops in the collaborative classroom? A study of video-based learning in higher education

  • Albó, Laia
  • Hernández Leo, Davinia
  • Moreno Oliver, Verónica
This paper explores how the use of smartphones vs. laptops influences students’ engagement, behaviour and experience watching academic videos in a collaborative classroom. Experiments were run in authentic teaching sessions with a total of 483 first-year higher education students. The methodology applied is a quasi-experimental design with post-test-only, being the independent variable, the device used to visualise the academic videos. Results indicate that the use of laptops has provided better results in terms of student’s engagement with the videos, their collaborative behaviour and satisfaction with the device. Hence, the findings of this research suggest that the type of mobile device used in activities that consider the use of videos in a collaborative class need to be carefully chosen to maximise the student’s comfortability – and in consequence, their engagement with the video-based learning activity and their positive behaviour and experience within the collaborative context., This work was supported by Ministerio de Ciencia y Tecnología [grant number MDM-2015-0502, TIN2014-53199-C3-3-R, TIN2017-85179-C3-3-R], Universitat Pompeu Fabra (UPF) [grant number PlaQUID2012], and RecerCaixa [grant number Communities of Teaching (CoT) Project].
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Design for collective intelligence: Pop-up communities in MOOCs

  • Garreta Domingo, Muriel
  • Sloep, Peter B.
  • Hernández Leo, Davinia
  • Mor, Yishay
Many current authors point toward the heightening of networked individualism and how this affects community creation and engagement. This trend poses strong challenges to the potential beneficial effects of collective intelligence. Education is one of the realms that can strongly suffer from this globalized individualism. Learning is deeply enhanced by social interactions and losing this social dimension will have long-lasting effects in future generations. Networked learning is also a by-product of our societal context, but not per se individual. Our paper presents a case—the HANDSON massive open online course (MOOC)—in which a purposely designed learning environment fosters the emergence of a kind of collective intelligence which, by the learners own accord, brings about a heightened sense of community. The MOOC’s design managed to enable individual learning paces without killing the social dimension. Thus, we argue that when learning together intentionally and informally in networked online environments, small and temporary communities (pop-up communities we call them) will form. This nascent sense of community is a first step that will ultimately contribute to the common good., We want to thank all facilitators for voluntarily donating their precious time to the HANDSON project, and to the HANDSON team as a whole for their productive collaboration. See the project website for details on the project and its participants http://handsonict.eu/. The HANDSON project (531086-LLP-1-2012-1-ES-KA3-KA3MP) has been funded with support from the Lifelong Learning Programme of the European Commission. This research has been partially supported by the Spanish Ministry of Economy and Competitiveness (project TIN2014-53199-C3-3-R) and Recer Caixa. DHR is a Serra Húnter fellow.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Teacher-led inquiry in technology-supported school communities

  • Michos, Konstantinos
  • Hernández Leo, Davinia
  • Albó, Laia
Learning design is a research field which studies how to best support teachers as designers of Technology Enhanced Learning (TEL) situations. Although substantial work has been done in the articulation of the learning design process, little is known about how learning designs are experienced by students and teachers, especially in the context of schools. This paper empirically examines if a teacher inquiry model, as a tool for systematic research by teachers into their own practice, facilitates the connection between the design and data‐informed reflection on TEL interventions in two school communities. High school teachers participated in a learning design professional development program supported by a web‐based community platform integrating a teacher inquiry tool (TILE). A multiple case study was conducted aimed at understanding: (a) current teacher practice and (b) teacher involvement in inquiry cycles of design and classroom implementations with technologies. Multiple data sources were used over a one year period including focus groups transcripts, teacher interview protocols, digital artifacts, and questionnaires. Sharing teacher‐led inquiries together with learning analytics was perceived as being useful for connecting pedagogical intentions with the evaluation of their enactment with learners, and this differed from their current practice. Teachers’ reflections about their designs focused on the time management of learning activities and their familiarity with the enactment and analytics tools. Results inform how technology can support teacher‐led inquiry and collective reflective practice in schools., This work has been partially funded by RecerCaixa (CoT project) and the Spanish Ministry of Economy and Competitiveness under MDM-2015-0502, TIN2014-53199-C3-3-R and TIN2017-85179-C3-3-R.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Human-centred design to empower ‘teachers as designers’

  • Garreta Domingo, Muriel
  • Sloep, Peter B.
  • Hernández Leo, Davinia
Educators of all sectors are learning designers, often unwittingly. To succeed as designers, they need to adopt a design mindset and acquire the skills needed to address the design challenges they encounter in their everyday practice. Human-centred design (HCD) provides professional designers with the methods needed to address complex problems. It emphasizes the human perspective throughout the design lifecycle and provides a practice-oriented approach, which naturally fits educators’ realities. This research reports the experiences of educators who used HCD to design ICT-based learning activities. A mixed-methods approach was used to gauge how participating educators experienced the design tasks. The perceived level of difficulty and value of the various methods varied, revealing significant differences between educators according to their level of knowledge of pedagogy frameworks. We discuss our findings from the vantage point of educators’ pedagogical beliefs and how experience shapes these. The results support the idea that HCD is a valuable framework for educators, one that may inform ongoing international efforts to shape a science and practice of learning design for teaching., This work has been partially supported by MDM-2015- 0502, TIN2014-53199-C3-3-R, TIN2017-85179-C3-3-R, RecerCaixa (CoT).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Knowledge-based design analytics for authoring courses with smart learning content

  • Albó, Laia
  • Barria-Pineda, Jordan
  • Brusilovsky, Peter
  • Hernández Leo, Davinia
Over the last 10 years, learning analytics have provided educators with both dashboards and tools to understand student behaviors within specific technological environments. However, there is a lack of work to support educators in making data-informed design decisions when designing a blended course and planning appropriate learning activities. In this paper, we introduce knowledge-based design analytics that uncover facets of the learning activities that are being created. A knowledge-based visualization is integrated into edCrumble, a (blended) learning design authoring tool. This new approach is explored in the context of a higher education programming course, where instructors design labs and home practice sessions with online smart learning content on a weekly basis. We performed a within-subjects user study to compare the use of the design tool both with and without visualization. We studied the differences in terms of cognitive load, controllability, confidence and ease of choice, design outcomes, and user actions within the system to compare both conditions with the objective of evaluating the impact of using design analytics during the decision-making phase of course design. Our results indicate that the use of a knowledge-based visualization allows the teachers to reduce the cognitive load (especially in terms of mental demand) and that it facilitates the choice of the most appropriate activities without affecting the overall design time. In conclusion, the use of knowledge-based design analytics improves the overall learning design quality and helps teachers avoid committing design errors., This work is a result of a collaboration within a mobility grant for research funded by the SEBAP, Societat Econòmica Barcelonesa d’Amics del País. This work has also been partially funded by NSF DRL 1740775, “la Caixa Foundation” (CoT project, 100010434) and FEDER, the National Research Agency of the Spanish Ministry of Science, Innovations and Universities MDM-2015-0502, TIN2014-53199-C3-3-R, TIN2017-85179-C3-3-R. D. Hernández-Leo acknowledges the support by ICREA under the ICREA Academia programme.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Evaluation to support learning design: Lessons learned in a teacher training MOOC

  • Garreta Domingo, Muriel
  • Hernández Leo, Davinia
  • Sloep, Peter B.
The design of learning opportunities is an integral part of the work of all educators. However, educators often lack the design skills and knowledge that professional designers have. One of these basic skills is related to the evaluation of produced artefacts, an essential step in all design frameworks. As a result, evaluation of learning activities falters due to this lack of knowledge and mindset. The present study analyses how in-service educators perceived and accomplished an evaluation activity aimed at promoting assessment prior to enactment. Heuristic evaluation is an inspection method considered to be one of the easiest to learn and yet is efficient, time and cost effective. These characteristics make it suitable for the design work of educators; which is, above all, practice-driven and practice-oriented. Results show that educators grasped the value of such an activity (solving possible second-order barriers) but struggled to understand how to do the specifics of the task (first-order barrier), which was to define their own set of educational heuristics. Based on the lessons learned, the paper finalises with a proposal for a design task to include evaluation to support learning design; empowering educators to assess both existing learning activities and ICT-tools as well as their own designs., The first and third authors want to thank the HANDSON team for their productive collaboration. The HANDSON project (531086-LLP-1-2012-1-ES- KA3-KA3MP) has been funded with support from the Lifelong Learning Programme of the European Commission. The second author’s contribution has been supported by TIN2014-53199-C3-3-R, MDM-2015-0502, and RecerCaixa CoT projects.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

CIDA: a collective inquiry framework to study and support teachers as designers in technological environments

  • Michos, Konstantinos
  • Hernández Leo, Davinia
The use of new technologies such as learning analytics by teachers is challenging due to the changes they bring to teachers’ practices and their pedagogical interventions. A design approach into teaching has been proposed for mapping pedagogy with technology to effectively integrate these changes. However, limited collective approaches exist in which teachers participate in professional communities to build knowledge for the design and implementation of learning scenarios with learning analytics. In this paper, we propose a framework (CIDA) for collective inquiry in such professional communities of teachers. Based on other collective inquiry frameworks for knowledge building, we propose three interconnected components: the inquiry process, the collective process, and technological support to facilitate and study teachers´ design practices in technological environments. We explain the framework with examples of implemented technologies and present empirical results of three cases; two High schools and a Massive Open Online Course (MOOC). Results inform further how our framework can be implemented in practice and its required supports to facilitate knowledge building for inquiry communities of teachers., DHL is a Serra Húnter Fellow. This work has been partially funded by “la Caixa Foundation” (CoT project, 100010434); by the European Regional Development Fund and the National Research Agency of the Spanish Ministry of Science, Innovation and Universities, under project grants TIN 2014-53199-C3-3-R, TIN2014-53199-C3-2-R, TIN 2017-85179-C3-2-R, TIN 2017-85179-C3-3-R, MDM-2015-0502
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Seeking reproducibility: assessing a multimodal study of the testing effect

  • Beardsley, Marc Yoshimi
  • Hernández Leo, Davinia
  • Ramírez, Rafael, 1966-
Low‐cost devices have widened the use of multimodal data in experiments providing a more complete picture of behavioural effects. However, the accurate collection and combination of multimodal and behavioural data in a manner that enables reproducibility is challenging and often requires researchers to refine their approaches. This paper presents a direct replication of a multimodal wordlist experiment. Specifically, we use a low‐cost Emotiv EPOC® to acquire electrophysiological measures of brain activity to investigate whether retrieval during learning facilitates the encoding of subsequent learning as measured by performance on recall tests and reflected by changes in alpha wave oscillations. Behavioural results of the wordlist experiment were replicated, but physiological results were not. We conclude the paper by highlighting the challenges faced in terms of replicating the previous work and in attempting to facilitate the reproducibility of our own experiment., This work has been partially supported by RecerCaixa (CoT project) and the Spanish Ministry of Economy and Competitiveness under MDM‐2015‐0502, TIN2014‐53199‐C3‐3‐R, TIN2017‐85179‐C3‐3‐R.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Sustaining continuous collaborative learning flows in MOOCs: orchestration agent approach

  • Amarasinghe, Ishari
  • Hernández Leo, Davinia
  • Manathunga, Kalpani
  • Jonsson, Anders, 1973-
Collaborative learning spaces deployed in Massive Open Online Courses (MOOCs) provide productive social learning opportunities. However, sustaining collaboration in these spaces is challenging. This paper provides a classification of MOOCs participants based on their behavior in a structured collaborative learning space. This analysis leads to requirements for new technological interventions to orchestrate collaborative learning flows in MOOCs. The paper proposes the design of an intelligent agent to address these requirements and reports a study which shows that the intervention of the proposed orchestration agent in a MOOC facilitates to maintain continuous yet meaningful collaboration learning flows., This research is funded by Spanish Ministry of Economy and Competitiveness (TIN2014-53199-C3-3-R, MDM-2015-0502) and RecerCaixa (COT Project).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Preface: connecting learning design and learning analytics

  • Hernández Leo, Davinia
  • Rodríguez-Triana, María Jesús
  • Salvador Inventado, Paul
  • Mor, Yishay
Prefaci del número 33 de la revista Interaction Design & Architecture(s) (IxD&aA), titulada "Connecting Learning Design and Learning Analytics"., This editorial work has been possible thanks to the support of the Spanish Ministry (TIN2014-53199-C3-3-R, MDM-2015-0502), RecerCaixa (CoT), National Science Foundation under DRL-1252297, and the European Union in the context of CEITER and Next-Lab (Horizon 2020 Research and Innovation Programme, grant agreements no. 669074 and 731685).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Conceptualising a visual representation model for MOOC-based blended learning designs

  • Albó, Laia
  • Hernández Leo, Davinia
This article reports on a study about how massive open online course (MOOC)-based blended learning designs can be visually represented to facilitate their comprehension and sharing. We carried out an iterative co-creation process with different stakeholders to conceptualise a visual learning design representation model within the context of blending MOOCs with face-to-face courses. The data analysed was derived from questionnaires and the generated representations. Results indicate that the representation enabled educators to easily visualise the overall structure of the learning designs and the relationships between the different design elements, providing a context for fostering reflection and decision-making during the planning of MOOC-based blended learning designs., The authors would like to thank all the participants of the workshops. This work has been partially funded by “la Caixa Foundation” (CoT project, 100010434) and FEDER, the National Research Agency of the Spanish Ministry of Science, Innovations and Universities MDM-2015-0502, TIN2014-53199-C3-3-R, TIN2017-85179-C3-3-R. Davinia Hernández-Leo (one of the authors) is a Serra Húnter Fellow.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Analytics for learning design: a layered framework and tools

  • Hernández Leo, Davinia
  • Martínez-Maldonado, Roberto
  • Pardo, Abelardo
  • Muñoz Cristóbal, Juan Alberto
  • Rodríguez-Triana, María Jesús
The field of learning design studies how to support teachers in devising suitable activities for their students to learn. The field of learning analytics explores how data about students’ interactions can be used to increase the understanding of learning experiences. Despite its clear synergy, there is only limited and fragmented work exploring the active role that data analytics can play in supporting design for learning. This paper builds on previous research to propose a framework (AL4LD) that articulates three layers of data analytics – learning analytics, design analytics and community analytics - to support informed decision-making in learning design. Additionally, a set of tools and experiences are described to illustrate how the different data analytics perspectives proposed by the framework can support learning design processes., This work has been partially supported by multiple projects, including RecerCaixa CoT, MDM-2015- 0502, TIN2014-53199-C3-3-R, TIN2017-85179-C3-3-R, TIN2014-53199-C3-2-R, TIN2017-85179- C3-2-R, TIN2011-28308-C03-02, VA082U16, FL100100203.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Supporting online collaborative design for teacher professional development

  • Marín, Victoria I.
  • Asensio-Pérez, Juan I.
  • Villagrá-Sobrino, Sara L.
  • Hernández Leo, Davinia
  • García-Sastre, Sara
This article describes a study of online collaborative design in the context of teacher professional development. Twenty-five teachers from different Spanish universities participated in the online course. The aim was to understand how to support teachers in interuniversity teams to collaborate fully online throughout the learning design process of a scenario based on their discipline integrating information and communications technology (ICT), an issue scarcely tackled in the literature. The described interpretive study, using mixed methods, explores the support of online co-design provided by a novel ICT community platform named ILDE (Integrated Learning Design Environment). Lessons drawn from the results can contribute to the improvement of online collaborative design processes in the context of teacher professional development., This work has been partially supported by the Spanish Ministry of Science, Innovation and Universities [project EDU2017-84223-R], the Spanish State Research Agency (AEI) [projects TIN2017-85179-C3-2-R and TIN2014-53199-C3-2-R, TIN2014-53199-C3-3-R and TIN2017-85179-C3-3-R], the Regional Government of Castilla y León and the European Regional Development Fund [project VA018U16], the European Commission [project 588438-EPP-1-2017-1-EL-EPPKA2-KA], the Spanish Ministry of Economy and Competitiveness [project MDM-2015-0502] and RecerCaixa (CoT).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Data-informed design parameters for adaptive collaborative scripting in across-spaces learning situations

  • Amarasinghe, Ishari
  • Hernández Leo, Davinia
  • Jonsson, Anders, 1973-
This study presents how predictive analytics can be used to inform the formulation of adaptive collaborative learning groups in the context of Computer Supported Collaborative Learning considering across-spaces learning situations. During the study we have collected data from different learning spaces which depicted both individual and collaborative learning activity engagement of students in two different learning contexts (namely the classroom learning and distance learning context) and attempted to predict individual student’s future collaborative learning activity participation in a pyramid-based collaborative learning activity using supervised machine learning techniques. We conducted experimental case studies in the classroom and in distance learning settings, in which real-time predictions of student’s future collaborative learning activity participation were used to formulate adaptive collaborative learner groups. Findings of the case studies showed that the data collected from across-spaces learning scenarios is informative when predicting future collaborative learning activity participation of students hence facilitating the formulation of adaptive collaborative group configurations that adapt to the activity participation differences of students in real-time. Limitations of the proposed approach and future research direction are illustrated., This work has been partially funded by FEDER, the National Research Agency of the Spanish Ministry of Science, Innovations and Universities MDM-2015-0502, TIN2014-53199-C3-3-R, TIN2017-85179-C3-3-R and “la Caixa Foundation” (CoT project, 100010434). DHL is a Serra Húnter Fellow.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

4FAD: A framework for mapping the evolution of artefacts in the learning design process

  • Muñoz Cristóbal, Juan Alberto
  • Hernández Leo, Davinia
  • Carvalho, Lucila
  • Martinez-Maldonado, Roberto
  • Thompson, Kate
  • Wardak, Dewa
  • Goodyear, Peter
A number of researchers have explored the role and nature of design in education, proposing a diverse array of life cycle models. Design plays subtly different roles in each of these models. The learning design research community is shifting its attention from the representation of pedagogical plans to considering design as an ongoing process. As a result, the study of the artefacts generated and used by educational designers is also changing: from a focus on the final designed artefact (the product of the design process) to the many artefacts generated and used by designers at different stages of the design process (e.g., sketches, reflections, drawings, or pictures). However, there is still a dearth of studies exploring the evolution of such artefacts throughout the learning design life cycle. A deeper understanding of these evolutionary processes is needed – to help smooth the transitions between stages in the life cycle. In this paper, we introduce the four-dimensional framework for artefacts in design (4FAD) to generate understanding and facilitate the mapping of the evolution of learning design artefacts. We illustrate the value of the framework by applying it in the analysis of an authentic design case., This research has been partially supported by the Spanish Projects TIN2011-28308-C03-02, TIN2014-53199-C3-2-R, TIN2014-53199-C3-3-R, TIN2017-85179-C3-2-R, TIN2017-85179-C3-3-R, MDM-2015-0502, RecerCaixa (CoT) and VA082U16, and by the Australian Research Council through Laureate grant FL100100203.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Ethics in educational technology research: Informing participants on data sharing risks

  • Beardsley, Marc Yoshimi
  • Santos Rodríguez, Patrícia
  • Hernández Leo, Davinia
  • Michos, Konstantinos
Participants in educational technology research regularly share personal data which carries with it risks. Informing participants of these data sharing risks is often only done so through text contained within a consent form. However, conceptualizations of data sharing risks and knowledge of responsible data management practices among teachers and learners may be impoverished—limiting the effectiveness of a consent form in communicating such risks in a manner that adequately supports participants in making informed decisions about sharing their data. At two high schools participating in an educational research project involving the use of technology in the classroom, we investigate teacher and student conceptions of data sharing risks and knowledge of responsible data management practices; and introduce a communication approach that attempts to better inform educational technology research participants of such risks. Results of this study suggest that most teachers have not received formal training related to responsibly managing data; and both teachers and students see the need for such training as they come to realize that their understanding of responsible data management is underdeveloped. Thus, efforts beyond solely explaining data sharing risks in an informed consent form may be needed in educational technology research to facilitate ethical self‐determination., This work has been partially funded by “la Caixa Foundation” (CoT project, 100010434) and FEDER, the National Research Agency of the Spanish Ministry of Science, Innovations and Universities MDM‐2015‐0502, TIN2014‐53199‐C3‐3‐R, TIN2017‐85179‐C3‐3‐R. DHL is a Serra Húnter Fellow.
Repositori Obert UdL
Publication Artículo científico (JournalArticle)

Supporting online collaborative design for teacher professional development

  • Marín, Victoria I.
  • Asensio Pérez, Juan I.
  • Villagrá Sobrino, Sara
  • Hernández Leo, Davinia
  • García Sastre, Sara
This paper describes a study on online collaborative design in the context of teacher professional development. 25 teachers from different Spanish universities and disciplines participated in the study. The aim was to understand how to support teachers in interuniversity teams to collaborate fully online along the learning design process of a discipline-based situation that integrates ICT, a problem scarcely tackled in the literature. The described interpretive study, using mixed methods, explores the support to online co-design provided by a novel ICT community platform named ILDE (Integrated Learning Design Environment). Lessons drawn from the results can contribute to the improvement of online collaborative design processes in the context of teacher professional development., This work has been partially supported by the Spanish Ministry of Science, Innovation and Universities [project EDU2017-84223-R], the Spanish State Research Agency (AEI) [projects TIN2017-85179-C3-2-R and TIN2014-53199-C3-2-R, TIN2014-53199-C3-3-R and TIN2017-85179-C3-3-R], the Regional Government of Castilla y León and the European Regional Development Fund [project VA018U16], the European Commission [project 588438-EPP-1-2017-1-EL-EPPKA2-KA], the Spanish Ministry of Economy and Competitiveness [project MDM-2015-0502] and RecerCaixa (CoT).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Smartphones or laptops in the collaborative classroom? A study of video-based learning in higher education

  • Albó, Laia
  • Hernández Leo, Davinia
  • Moreno Oliver, Verónica
This paper explores how the use of smartphones vs. laptops influences students’ engagement, behaviour and experience watching academic videos in a collaborative classroom. Experiments were run in authentic teaching sessions with a total of 483 first-year higher education students. The methodology applied is a quasi-experimental design with post-test-only, being the independent variable, the device used to visualise the academic videos. Results indicate that the use of laptops has provided better results in terms of student’s engagement with the videos, their collaborative behaviour and satisfaction with the device. Hence, the findings of this research suggest that the type of mobile device used in activities that consider the use of videos in a collaborative class need to be carefully chosen to maximise the student’s comfortability – and in consequence, their engagement with the video-based learning activity and their positive behaviour and experience within the collaborative context., This work was supported by Ministerio de Ciencia y Tecnología [grant number MDM-2015-0502, TIN2014-53199-C3-3-R, TIN2017-85179-C3-3-R], Universitat Pompeu Fabra (UPF) [grant number PlaQUID2012], and RecerCaixa [grant number Communities of Teaching (CoT) Project].
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

An integrated environment for learning design

  • Hernández Leo, Davinia
  • Asensio-Pérez, Juan I.
  • Derntl, Michael
  • Pozzi, Francesca
  • Chacón Pérez, Jonathan, 1986-
  • Prieto, Luis P.
  • Persico, Donatella
Learning design tools aim at supporting practitioners in their task of creating more innovative and effective computer-supported learning situations. Despite there being a myriad of proposed tools, their use presents challenges that recent studies link with practitioners' varied pedagogical approaches and context restrictions, as well as with barriers to practical application derived from the fact that most tools only cover limited functionality and do not support cooperation between practitioners. In this paper we investigate whether it is possible to provide a flexible community system that supports multiple learning design tasks. We propose an Integrated Learning Design Environment (ILDE), which is a networked system integrating collaboration functions, design editors and middleware that enables deployment of the designed learning situations into Virtual Learning Environments. We describe the iterative user-centered process adopted in the design of ILDE as well as its architecture. The architecture is implemented to show its feasibility and that it is capable of providing the targeted functionality. We also present the results of its use in training workshops with 148 practitioners from five different institutions in vocational training, higher and adult education. Some of the learning designs were deployed in VLEs and enacted with students in real learning situations., This work has been partially funded by the EC, EACEA, METIS Project 531262-LLP-2012-ES-KA3-KA3MP, with additional support by TIN2014-53199-C3-3-R, TIN2017-85179-C3-3-R, MDM-2015-0502, RecerCaixa CoT, TIN2014-53199-C3-2-R, TIN2017-85179-C3-2-R, VA082U16 and a Marie Curie Fellowship (MIOCTI, FP7-PEOPLE-2012-IEF project no. 327384).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Human-centred design to empower ‘teachers as designers’

  • Garreta Domingo, Muriel
  • Sloep, Peter B.
  • Hernández Leo, Davinia
Educators of all sectors are learning designers, often unwittingly. To succeed as designers, they need to adopt a design mindset and acquire the skills needed to address the design challenges they encounter in their everyday practice. Human-centred design (HCD) provides professional designers with the methods needed to address complex problems. It emphasizes the human perspective throughout the design lifecycle and provides a practice-oriented approach, which naturally fits educators’ realities. This research reports the experiences of educators who used HCD to design ICT-based learning activities. A mixed-methods approach was used to gauge how participating educators experienced the design tasks. The perceived level of difficulty and value of the various methods varied, revealing significant differences between educators according to their level of knowledge of pedagogy frameworks. We discuss our findings from the vantage point of educators’ pedagogical beliefs and how experience shapes these. The results support the idea that HCD is a valuable framework for educators, one that may inform ongoing international efforts to shape a science and practice of learning design for teaching., This work has been partially supported by MDM-2015- 0502, TIN2014-53199-C3-3-R, TIN2017-85179-C3-3-R, RecerCaixa (CoT).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

When we talk about older people in HCI, who are we talking about? Towards a ‘turn to community’ in the design of technologies for a growing ageing population

  • Righi, Valeria
  • Sayago Barrantes, Sergio
  • Blat, Josep
This paper addresses a number of challenges HCI designers and researchers deal with when designing digital technologies for older people. We conducted a 5-year research-through-design study, which combined ethnography, participatory design and real-life evaluation, to explore the design and use of technologies aimed to enhance the social life of older people. The paper explores widespread assumptions about the social category of older people and the meaning of technologies for them. This paper argues that assuming that (i) older people have unique needs and interests that set them apart from other user groups and (ii) the meaning of technologies remains unaltered on completion of co-design activities are both problematic, because they do not consider carefully enough the sociocultural contexts in which older people interact and engage in their daily lives. The paper posits that the meaning of technologies for older people and their own identities as individuals of a certain age are shaped within situated communities. Thus, technologies designed ‘for older people’ should be designed to meet situated and dynamic needs/interests of the communities (and not only of care) to which they belong. We draw upon our findings and other situated epistemological discourses in HCI to introduce a different perspective, a turn to community, in the design of technologies for an ever-increasing ageing population., This research has been partially funded by the Spanish Ministry of Economy and Competitiveness (RESET TIN2014-53199-C3-3-R), the EU through Life2.0 (CIP ICT PSP-2009-4-270965), and Fundación General CSIC and Obra Social ‘‘la Caixa’’ through the WorthPlay project.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Co-designing with a community of older learners for over 10 years by moving user-driven participation from the margin to the centre

  • Righi, Valeria
  • Sayago Barrantes, Sergio
  • Rosales, Andrea
  • Ferreira, Susan M.
  • Blat, Josep
This paper addresses a gap in the Participatory Design (PD) literature, wherein more research on the long-term impact of design projects is warranted. This paper reflects on a 10-year study that intertwined ethnography and 2 PD projects in a community of older learners. Although the goal of our study was to design new digital technologies, the process of designing them presented us with opportunities that gave rise to new non-digital practices, which turned out to be the legacy and most significant outcomes of the PD projects. This result invited us to review the trajectory that led to these outcomes. Our analysis shows that the most important legacy aspect of the projects arose from unexpected forms of user—driven participation that we allowed to co-exist together with those practices more related to the design goals of the PD projects. Drawing upon our results, this paper posits that engagement with PD participants that unfolds over an extended period of time is instrumental in facilitating the development of participation, understanding more deeply long-lasting project outcomes, and legitimising forms of participation that are not directly related to project/design goals., We are indebted to all the participants who took part in our study for having allowed us to become members and feel part of their communities. This research has been partially funded by the Spanish Ministry of Economy and Competitiveness (RESET TIN2014-53199-C3- 3-R ), the EU (CIP ICT PSP-2009-4-270965 ), and Fundación General CSIC and Obra Social ‘‘la Caixa’’. The authors also acknowledge the suggestions made by the reviewers.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

An actionable orchestration dashboard to enhance collaboration in the classroom

  • Amarasinghe, Ishari
  • Hernández Leo, Davinia
  • Michos, Konstantinos
  • Vujovic, Milica
The orchestration of collaborative learning activities in technology-enhanced classrooms has become a nontrivial endeavor for educators. Depending on the behaviors and needs of students that emerge in real educational situations, educators may need to orchestrate activity adaptations on the fly. These adaptations may range from the provision of additional scaffolding by the educator (e.g., the educator's participation in a group discussion) to a change in the planned pedagogical scenario (e.g., the duration). This article aims to contribute to the orchestration of technology-mediated collaborative learning sessions in a classroom context. We present the design, implementation, and evaluation of a teacher-facing dashboard that supports teachers in orchestrating scripted collaboration. Evaluation studies were conducted in 16 classroom sessions. The findings indicate that teachers found the information on the dashboard to be actionable and help facilitate just in time support to student groups., This work has been partially funded by "la Caixa Foundation" (CoT project, 100010434) and the National Research Agency of the Spanish Ministry of Science, Innovations and Universities MDM-2015- 0502, TIN2014-53199-C3-3-R, TIN2017-85179-C3-3-R.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Conceptualising a visual representation model for MOOC-based blended learning designs

  • Albó, Laia
  • Hernández Leo, Davinia
This article reports on a study about how massive open online course (MOOC)-based blended learning designs can be visually represented to facilitate their comprehension and sharing. We carried out an iterative co-creation process with different stakeholders to conceptualise a visual learning design representation model within the context of blending MOOCs with face-to-face courses. The data analysed was derived from questionnaires and the generated representations. Results indicate that the representation enabled educators to easily visualise the overall structure of the learning designs and the relationships between the different design elements, providing a context for fostering reflection and decision-making during the planning of MOOC-based blended learning designs., The authors would like to thank all the participants of the workshops. This work has been partially funded by “la Caixa Foundation” (CoT project, 100010434) and FEDER, the National Research Agency of the Spanish Ministry of Science, Innovations and Universities MDM-2015-0502, TIN2014-53199-C3-3-R, TIN2017-85179-C3-3-R. Davinia Hernández-Leo (one of the authors) is a Serra Húnter Fellow.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Teacher-led inquiry in technology-supported school communities

  • Michos, Konstantinos
  • Hernández Leo, Davinia
  • Albó, Laia
Learning design is a research field which studies how to best support teachers as designers of Technology Enhanced Learning (TEL) situations. Although substantial work has been done in the articulation of the learning design process, little is known about how learning designs are experienced by students and teachers, especially in the context of schools. This paper empirically examines if a teacher inquiry model, as a tool for systematic research by teachers into their own practice, facilitates the connection between the design and data‐informed reflection on TEL interventions in two school communities. High school teachers participated in a learning design professional development program supported by a web‐based community platform integrating a teacher inquiry tool (TILE). A multiple case study was conducted aimed at understanding: (a) current teacher practice and (b) teacher involvement in inquiry cycles of design and classroom implementations with technologies. Multiple data sources were used over a one year period including focus groups transcripts, teacher interview protocols, digital artifacts, and questionnaires. Sharing teacher‐led inquiries together with learning analytics was perceived as being useful for connecting pedagogical intentions with the evaluation of their enactment with learners, and this differed from their current practice. Teachers’ reflections about their designs focused on the time management of learning activities and their familiarity with the enactment and analytics tools. Results inform how technology can support teacher‐led inquiry and collective reflective practice in schools., This work has been partially funded by RecerCaixa (CoT project) and the Spanish Ministry of Economy and Competitiveness under MDM-2015-0502, TIN2014-53199-C3-3-R and TIN2017-85179-C3-3-R.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Supporting online collaborative design for teacher professional development

  • Marín, Victoria I.
  • Asensio Pérez, Juan I.
  • Villagrá Sobrino, Sara
  • Hernández Leo, Davinia
  • García Sastre, Sara
This paper describes a study on online collaborative design in the context of teacher professional development. 25 teachers from different Spanish universities and disciplines participated in the study. The aim was to understand how to support teachers in interuniversity teams to collaborate fully online along the learning design process of a discipline-based situation that integrates ICT, a problem scarcely tackled in the literature. The described interpretive study, using mixed methods, explores the support to online co-design provided by a novel ICT community platform named ILDE (Integrated Learning Design Environment). Lessons drawn from the results can contribute to the improvement of online collaborative design processes in the context of teacher professional development., This work has been partially supported by the Spanish Ministry of Science, Innovation and Universities [project EDU2017-84223-R], the Spanish State Research Agency (AEI) [projects TIN2017-85179-C3-2-R and TIN2014-53199-C3-2-R, TIN2014-53199-C3-3-R and TIN2017-85179-C3-3-R], the Regional Government of Castilla y León and the European Regional Development Fund [project VA018U16], the European Commission [project 588438-EPP-1-2017-1-EL-EPPKA2-KA], the Spanish Ministry of Economy and Competitiveness [project MDM-2015-0502] and RecerCaixa (CoT).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Ethics in educational technology research: Informing participants on data sharing risks

  • Beardsley, Marc Yoshimi
  • Santos Rodríguez, Patrícia
  • Hernández Leo, Davinia
  • Michos, Konstantinos
Participants in educational technology research regularly share personal data which carries with it risks. Informing participants of these data sharing risks is often only done so through text contained within a consent form. However, conceptualizations of data sharing risks and knowledge of responsible data management practices among teachers and learners may be impoverished—limiting the effectiveness of a consent form in communicating such risks in a manner that adequately supports participants in making informed decisions about sharing their data. At two high schools participating in an educational research project involving the use of technology in the classroom, we investigate teacher and student conceptions of data sharing risks and knowledge of responsible data management practices; and introduce a communication approach that attempts to better inform educational technology research participants of such risks. Results of this study suggest that most teachers have not received formal training related to responsibly managing data; and both teachers and students see the need for such training as they come to realize that their understanding of responsible data management is underdeveloped. Thus, efforts beyond solely explaining data sharing risks in an informed consent form may be needed in educational technology research to facilitate ethical self‐determination., This work has been partially funded by “la Caixa Foundation” (CoT project, 100010434) and FEDER, the National Research Agency of the Spanish Ministry of Science, Innovations and Universities MDM‐2015‐0502, TIN2014‐53199‐C3‐3‐R, TIN2017‐85179‐C3‐3‐R. DHL is a Serra Húnter Fellow.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Knowledge-based design analytics for authoring courses with smart learning content

  • Albó, Laia
  • Barria-Pineda, Jordan
  • Brusilovsky, Peter
  • Hernández Leo, Davinia
Over the last 10 years, learning analytics have provided educators with both dashboards and tools to understand student behaviors within specific technological environments. However, there is a lack of work to support educators in making data-informed design decisions when designing a blended course and planning appropriate learning activities. In this paper, we introduce knowledge-based design analytics that uncover facets of the learning activities that are being created. A knowledge-based visualization is integrated into edCrumble, a (blended) learning design authoring tool. This new approach is explored in the context of a higher education programming course, where instructors design labs and home practice sessions with online smart learning content on a weekly basis. We performed a within-subjects user study to compare the use of the design tool both with and without visualization. We studied the differences in terms of cognitive load, controllability, confidence and ease of choice, design outcomes, and user actions within the system to compare both conditions with the objective of evaluating the impact of using design analytics during the decision-making phase of course design. Our results indicate that the use of a knowledge-based visualization allows the teachers to reduce the cognitive load (especially in terms of mental demand) and that it facilitates the choice of the most appropriate activities without affecting the overall design time. In conclusion, the use of knowledge-based design analytics improves the overall learning design quality and helps teachers avoid committing design errors., This work is a result of a collaboration within a mobility grant for research funded by the SEBAP, Societat Econòmica Barcelonesa d’Amics del País. This work has also been partially funded by NSF DRL 1740775, “la Caixa Foundation” (CoT project, 100010434) and FEDER, the National Research Agency of the Spanish Ministry of Science, Innovations and Universities MDM-2015-0502, TIN2014-53199-C3-3-R, TIN2017-85179-C3-3-R. D. Hernández-Leo acknowledges the support by ICREA under the ICREA Academia programme.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Supporting online collaborative design for teacher professional development

  • Marín, Victoria I.
  • Asensio-Pérez, Juan I.
  • Villagrá-Sobrino, Sara L.
  • Hernández Leo, Davinia
  • García-Sastre, Sara
This article describes a study of online collaborative design in the context of teacher professional development. Twenty-five teachers from different Spanish universities participated in the online course. The aim was to understand how to support teachers in interuniversity teams to collaborate fully online throughout the learning design process of a scenario based on their discipline integrating information and communications technology (ICT), an issue scarcely tackled in the literature. The described interpretive study, using mixed methods, explores the support of online co-design provided by a novel ICT community platform named ILDE (Integrated Learning Design Environment). Lessons drawn from the results can contribute to the improvement of online collaborative design processes in the context of teacher professional development., This work has been partially supported by the Spanish Ministry of Science, Innovation and Universities [project EDU2017-84223-R], the Spanish State Research Agency (AEI) [projects TIN2017-85179-C3-2-R and TIN2014-53199-C3-2-R, TIN2014-53199-C3-3-R and TIN2017-85179-C3-3-R], the Regional Government of Castilla y León and the European Regional Development Fund [project VA018U16], the European Commission [project 588438-EPP-1-2017-1-EL-EPPKA2-KA], the Spanish Ministry of Economy and Competitiveness [project MDM-2015-0502] and RecerCaixa (CoT).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Evaluation to support learning design: Lessons learned in a teacher training MOOC

  • Garreta Domingo, Muriel
  • Hernández Leo, Davinia
  • Sloep, Peter B.
The design of learning opportunities is an integral part of the work of all educators. However, educators often lack the design skills and knowledge that professional designers have. One of these basic skills is related to the evaluation of produced artefacts, an essential step in all design frameworks. As a result, evaluation of learning activities falters due to this lack of knowledge and mindset. The present study analyses how in-service educators perceived and accomplished an evaluation activity aimed at promoting assessment prior to enactment. Heuristic evaluation is an inspection method considered to be one of the easiest to learn and yet is efficient, time and cost effective. These characteristics make it suitable for the design work of educators; which is, above all, practice-driven and practice-oriented. Results show that educators grasped the value of such an activity (solving possible second-order barriers) but struggled to understand how to do the specifics of the task (first-order barrier), which was to define their own set of educational heuristics. Based on the lessons learned, the paper finalises with a proposal for a design task to include evaluation to support learning design; empowering educators to assess both existing learning activities and ICT-tools as well as their own designs., The first and third authors want to thank the HANDSON team for their productive collaboration. The HANDSON project (531086-LLP-1-2012-1-ES- KA3-KA3MP) has been funded with support from the Lifelong Learning Programme of the European Commission. The second author’s contribution has been supported by TIN2014-53199-C3-3-R, MDM-2015-0502, and RecerCaixa CoT projects.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Seeking reproducibility: assessing a multimodal study of the testing effect

  • Beardsley, Marc Yoshimi
  • Hernández Leo, Davinia
  • Ramírez, Rafael, 1966-
Low‐cost devices have widened the use of multimodal data in experiments providing a more complete picture of behavioural effects. However, the accurate collection and combination of multimodal and behavioural data in a manner that enables reproducibility is challenging and often requires researchers to refine their approaches. This paper presents a direct replication of a multimodal wordlist experiment. Specifically, we use a low‐cost Emotiv EPOC® to acquire electrophysiological measures of brain activity to investigate whether retrieval during learning facilitates the encoding of subsequent learning as measured by performance on recall tests and reflected by changes in alpha wave oscillations. Behavioural results of the wordlist experiment were replicated, but physiological results were not. We conclude the paper by highlighting the challenges faced in terms of replicating the previous work and in attempting to facilitate the reproducibility of our own experiment., This work has been partially supported by RecerCaixa (CoT project) and the Spanish Ministry of Economy and Competitiveness under MDM‐2015‐0502, TIN2014‐53199‐C3‐3‐R, TIN2017‐85179‐C3‐3‐R.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Supporting collaborative design activity in a multi-user digital design ecology

  • Martinez-Maldonado, Roberto
  • Goodyear, Peter
  • Carvalho, Lucila
  • Thompson, Kate
  • Hernández Leo, Davinia
  • Dimitriadis, Yannis
  • Prieto, Luis P.
  • Wardak, Dewa
Across a broad range of design professions, there has been extensive research on design practices and considerable progress in creating new computer-based systems that support design work. Our research is focused on educational/instructional design for students' learning. In this sub-field, progress has been more limited. In particular, neither research nor systems development have paid much attention to the fact that design is becoming a more collaborative endeavor. This paper reports the latest research outcomes from R&D in the Educational Design Studio (EDS), a facility developed iteratively over four years to support and understand collaborative, real-time, co-present design work. The EDS serves to (i) enhance our scientific understanding of design processes and design cognition and (ii) provide insights into how designers' work can be improved through appropriate technological support. In the study presented here, we introduced a complex, multi-user, digital design tool into the existing ecology of tools and resources available in the EDS. We analysed the activity of four pairs of ‘teacher-designers’ during a design task. We identified different behaviors - in reconfiguring the task, the working methods and toolset usage. Our data provide new insights about the affordances of different digital and analogue design surfaces used in the Studio., This work was funded by the Australian Research Council (Laureate Fellowship Grant FL100100203) and by the Spanish Ministry (TIN2014-53199-C3-3-R, MDM-2015-0502). Studies presented in this paper were conducted under human ethics protocol number 2012/2794, entitled “Learning, technology and design: architectures for productive networked learning - Using complementary methods of analysis".
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Sustaining continuous collaborative learning flows in MOOCs: orchestration agent approach

  • Amarasinghe, Ishari
  • Hernández Leo, Davinia
  • Manathunga, Kalpani
  • Jonsson, Anders, 1973-
Collaborative learning spaces deployed in Massive Open Online Courses (MOOCs) provide productive social learning opportunities. However, sustaining collaboration in these spaces is challenging. This paper provides a classification of MOOCs participants based on their behavior in a structured collaborative learning space. This analysis leads to requirements for new technological interventions to orchestrate collaborative learning flows in MOOCs. The paper proposes the design of an intelligent agent to address these requirements and reports a study which shows that the intervention of the proposed orchestration agent in a MOOC facilitates to maintain continuous yet meaningful collaboration learning flows., This research is funded by Spanish Ministry of Economy and Competitiveness (TIN2014-53199-C3-3-R, MDM-2015-0502) and RecerCaixa (COT Project).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Preface: connecting learning design and learning analytics

  • Hernández Leo, Davinia
  • Rodríguez-Triana, María Jesús
  • Salvador Inventado, Paul
  • Mor, Yishay
Prefaci del número 33 de la revista Interaction Design & Architecture(s) (IxD&aA), titulada "Connecting Learning Design and Learning Analytics"., This editorial work has been possible thanks to the support of the Spanish Ministry (TIN2014-53199-C3-3-R, MDM-2015-0502), RecerCaixa (CoT), National Science Foundation under DRL-1252297, and the European Union in the context of CEITER and Next-Lab (Horizon 2020 Research and Innovation Programme, grant agreements no. 669074 and 731685).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

4FAD: A framework for mapping the evolution of artefacts in the learning design process

  • Muñoz Cristóbal, Juan Alberto
  • Hernández Leo, Davinia
  • Carvalho, Lucila
  • Martinez-Maldonado, Roberto
  • Thompson, Kate
  • Wardak, Dewa
  • Goodyear, Peter
A number of researchers have explored the role and nature of design in education, proposing a diverse array of life cycle models. Design plays subtly different roles in each of these models. The learning design research community is shifting its attention from the representation of pedagogical plans to considering design as an ongoing process. As a result, the study of the artefacts generated and used by educational designers is also changing: from a focus on the final designed artefact (the product of the design process) to the many artefacts generated and used by designers at different stages of the design process (e.g., sketches, reflections, drawings, or pictures). However, there is still a dearth of studies exploring the evolution of such artefacts throughout the learning design life cycle. A deeper understanding of these evolutionary processes is needed – to help smooth the transitions between stages in the life cycle. In this paper, we introduce the four-dimensional framework for artefacts in design (4FAD) to generate understanding and facilitate the mapping of the evolution of learning design artefacts. We illustrate the value of the framework by applying it in the analysis of an authentic design case., This research has been partially supported by the Spanish Projects TIN2011-28308-C03-02, TIN2014-53199-C3-2-R, TIN2014-53199-C3-3-R, TIN2017-85179-C3-2-R, TIN2017-85179-C3-3-R, MDM-2015-0502, RecerCaixa (CoT) and VA082U16, and by the Australian Research Council through Laureate grant FL100100203.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

EdCrumble, a data-enriched visual authoring design tool for blended learning

  • Albó, Laia
  • Hernández Leo, Davinia
This paper presents an evaluation of edCrumble, a blended learning authoring tool for teachers. The tool visually represents learning designs and integrates data analytics to scaffold teacher design decisions. In addition to assessing the usability of edCrumble using UMUX questionnaire, analyses of participant views on the advantages and disadvantages of the tool in comparison to traditional course planning approaches and participant-elicited factors that can facilitate or hinder adoption of the tool are presented. Three evaluation workshops were held with teachers of different backgrounds, educational stakeholders, and students. Data-gathering instruments included interviews and questionnaires to collect qualitative and quantitative data. Further, design artifacts resulting from the workshops were analysed. The evaluation shows that the tool possesses specific features that facilitate the representation of and support for designing blended learning, uncovers the factors that may promote or inhibit its adoption, exposes its connection with solving actual educational challenges, and reveals the strengths and weaknesses in the tools usability., This work was supported by the Spanish Ministry of Science (TIN2014-53199-C3-3-R and TIN2017-85179-C3-3-R), the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502), RecerCaixa (CoT project) and the UPF PlaQUID2012. D.H.L acknowledges the support by ICREA under the ICREA Academia programme. (Corresponding author: Laia Albó).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

CIDA: a collective inquiry framework to study and support teachers as designers in technological environments

  • Michos, Konstantinos
  • Hernández Leo, Davinia
The use of new technologies such as learning analytics by teachers is challenging due to the changes they bring to teachers’ practices and their pedagogical interventions. A design approach into teaching has been proposed for mapping pedagogy with technology to effectively integrate these changes. However, limited collective approaches exist in which teachers participate in professional communities to build knowledge for the design and implementation of learning scenarios with learning analytics. In this paper, we propose a framework (CIDA) for collective inquiry in such professional communities of teachers. Based on other collective inquiry frameworks for knowledge building, we propose three interconnected components: the inquiry process, the collective process, and technological support to facilitate and study teachers´ design practices in technological environments. We explain the framework with examples of implemented technologies and present empirical results of three cases; two High schools and a Massive Open Online Course (MOOC). Results inform further how our framework can be implemented in practice and its required supports to facilitate knowledge building for inquiry communities of teachers., DHL is a Serra Húnter Fellow. This work has been partially funded by “la Caixa Foundation” (CoT project, 100010434); by the European Regional Development Fund and the National Research Agency of the Spanish Ministry of Science, Innovation and Universities, under project grants TIN 2014-53199-C3-3-R, TIN2014-53199-C3-2-R, TIN 2017-85179-C3-2-R, TIN 2017-85179-C3-3-R, MDM-2015-0502
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Data-informed design parameters for adaptive collaborative scripting in across-spaces learning situations

  • Amarasinghe, Ishari
  • Hernández Leo, Davinia
  • Jonsson, Anders, 1973-
This study presents how predictive analytics can be used to inform the formulation of adaptive collaborative learning groups in the context of Computer Supported Collaborative Learning considering across-spaces learning situations. During the study we have collected data from different learning spaces which depicted both individual and collaborative learning activity engagement of students in two different learning contexts (namely the classroom learning and distance learning context) and attempted to predict individual student’s future collaborative learning activity participation in a pyramid-based collaborative learning activity using supervised machine learning techniques. We conducted experimental case studies in the classroom and in distance learning settings, in which real-time predictions of student’s future collaborative learning activity participation were used to formulate adaptive collaborative learner groups. Findings of the case studies showed that the data collected from across-spaces learning scenarios is informative when predicting future collaborative learning activity participation of students hence facilitating the formulation of adaptive collaborative group configurations that adapt to the activity participation differences of students in real-time. Limitations of the proposed approach and future research direction are illustrated., This work has been partially funded by FEDER, the National Research Agency of the Spanish Ministry of Science, Innovations and Universities MDM-2015-0502, TIN2014-53199-C3-3-R, TIN2017-85179-C3-3-R and “la Caixa Foundation” (CoT project, 100010434). DHL is a Serra Húnter Fellow.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Design for collective intelligence: Pop-up communities in MOOCs

  • Garreta Domingo, Muriel
  • Sloep, Peter B.
  • Hernández Leo, Davinia
  • Mor, Yishay
Many current authors point toward the heightening of networked individualism and how this affects community creation and engagement. This trend poses strong challenges to the potential beneficial effects of collective intelligence. Education is one of the realms that can strongly suffer from this globalized individualism. Learning is deeply enhanced by social interactions and losing this social dimension will have long-lasting effects in future generations. Networked learning is also a by-product of our societal context, but not per se individual. Our paper presents a case—the HANDSON massive open online course (MOOC)—in which a purposely designed learning environment fosters the emergence of a kind of collective intelligence which, by the learners own accord, brings about a heightened sense of community. The MOOC’s design managed to enable individual learning paces without killing the social dimension. Thus, we argue that when learning together intentionally and informally in networked online environments, small and temporary communities (pop-up communities we call them) will form. This nascent sense of community is a first step that will ultimately contribute to the common good., We want to thank all facilitators for voluntarily donating their precious time to the HANDSON project, and to the HANDSON team as a whole for their productive collaboration. See the project website for details on the project and its participants http://handsonict.eu/. The HANDSON project (531086-LLP-1-2012-1-ES-KA3-KA3MP) has been funded with support from the Lifelong Learning Programme of the European Commission. This research has been partially supported by the Spanish Ministry of Economy and Competitiveness (project TIN2014-53199-C3-3-R) and Recer Caixa. DHR is a Serra Húnter fellow.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Analytics for learning design: a layered framework and tools

  • Hernández Leo, Davinia
  • Martínez-Maldonado, Roberto
  • Pardo, Abelardo
  • Muñoz Cristóbal, Juan Alberto
  • Rodríguez-Triana, María Jesús
The field of learning design studies how to support teachers in devising suitable activities for their students to learn. The field of learning analytics explores how data about students’ interactions can be used to increase the understanding of learning experiences. Despite its clear synergy, there is only limited and fragmented work exploring the active role that data analytics can play in supporting design for learning. This paper builds on previous research to propose a framework (AL4LD) that articulates three layers of data analytics – learning analytics, design analytics and community analytics - to support informed decision-making in learning design. Additionally, a set of tools and experiences are described to illustrate how the different data analytics perspectives proposed by the framework can support learning design processes., This work has been partially supported by multiple projects, including RecerCaixa CoT, MDM-2015- 0502, TIN2014-53199-C3-3-R, TIN2017-85179-C3-3-R, TIN2014-53199-C3-2-R, TIN2017-85179- C3-2-R, TIN2011-28308-C03-02, VA082U16, FL100100203.