Pasar al contenido principal

Rethinking randomised controlled trials: Causal claims, the reproducibility crisis, and combining methods

RYC2020-029424-I

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal de Promoción del Talento y su Empleabilidad en I+D+i
Subprograma Subprograma Estatal de Incorporación
Convocatoria Ramón y Cajal
Año convocatoria 2020
Plan Estatal 2017-2020 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario AGENCIA ESTATAL CONSEJO SUPERIOR DE INVESTIGACIONES CIENTIFICAS (CSIC)

Resultados relacionados

Resultados totales (Incluyendo duplicados): 13
Encontrada(s) 1 página(s)

DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Differences in collaboration structures and impact among prominent researchers in Europe and North America

  • Danús, Lluís
  • Muntaner, Carles
  • Krauss, Alexander
  • Sales-Pardo, Marta
  • Guimerà, Roger
Scientists collaborate through intricate networks, which impact the quality and scope of their research. At the same time, funding and institutional arrangements, as well as scientific and political cultures, affect the structure of collaboration networks. Since such arrangements and cultures differ across regions in the world in systematic ways, we surmise that collaboration networks and impact should also differ systematically across regions. To test this, we compare the structure of collaboration networks among prominent researchers in North America and Europe. We find that prominent researchers in Europe establish denser collaboration networks, whereas those in North America establish more decentralized networks. We also find that the impact of the publications of prominent researchers in North America is significantly higher than for those in Europe, both when they collaborate with other prominent researchers and when they do not. Although Europeans collaborate with other prominent researchers more often, which increases their impact, we also find that repeated collaboration among prominent researchers decreases the synergistic effect of collaborating., This project has received funding from the Spanish Ministerio de Economía y Competitividad (PID2019-106811GB-C31) and from the Government of Catalonia (2017SGR-896). AK received funding from the Ministry of Science and Innovation of the Government of Spain (grant RYC2020-029424-I).
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Early-career factors largely determine the future impact of prominent researchers: evidence across eight scientific fields

  • Krauss, Alexander
  • Danús, Lluís
  • Sales-Pardo, Marta
Can we help predict the future impact of researchers using early-career factors? We analyze early-career factors of the world's 100 most prominent researchers across 8 scientific fields and identify four key drivers in researchers' initial career: working at a top 25 ranked university, publishing a paper in a top 5 ranked journal, publishing most papers in top quartile (high-impact) journals and co-authoring with other prominent researchers in their field. We find that over 95% of prominent researchers across multiple fields had at least one of these four features in the first 5 years of their career. We find that the most prominent scientists who had an early career advantage in terms of citations and h-index are more likely to have had all four features, and that this advantage persists throughout their career after 10, 15 and 20 years. Our findings show that these few early-career factors help predict researchers' impact later in their careers. Our research thus points to the need to enhance fairness and career mobility among scientists who have not had a jump start early on., AK received funding from the Ministry of Science and Innovation of the Government of Spain (grant RYC2020-029424-I). MS-P acknowledges support from PID2019-106811GB-C31, from MCIN/ AEI/ 10.13039/ 501100011033, and from the Government of Catalonia (2017SGR-896)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Homo methodologicus and the origin of science and civilisation

  • Krauss, Alexander
Few things have impacted our lives as much as science and technology, but how we developed science and civilisation is one of the most challenging questions that has not yet been well explained. Attempting to identify the central driver, leading scientists have highlighted the role of culture, cooperation and geography. They focus thus on broad factors that are important basic preconditions but that we cannot directly influence. To better address the question, this paper integrates evidence from evolutionary biology, cognitive science, methodology, archaeology and anthropology. The paper identifies 9 main preconditions necessary for contemporary science, which include 6 main preconditions for civilisation. Using a kind of quasi-experimental research design we observe that some cultures (experimental groups) met the preconditions while other cultures (control groups) did not. Among the preconditions, we explain how our mind's evolved methodological abilities (to observe, solve problems and experiment) have directly enabled acquiring knowledge about the world and collectively developing increasingly sophisticated methods (such as mathematics and more systematic experimentation) that have enabled science and civilisation. We have driven the major revolutions throughout our history - the palaeolithic technological and agricultural revolutions and later the so-called scientific, industrial and digital revolutions - by using our methodological abilities in new ways and developing new methods and tools, i.e. through methodological revolutions. Viewing our methods as the main mechanism through which we have directly developed scientific and technological knowledge, and thus science and civilisation, provides a new framework for understanding science and the history of science. Viewing humans as homo methodologicus, using an expanding methodological toolbox, provides a nuanced explanation of how we have been directly able to meet our needs, solve problems and develop vast bodies of technological and scientific knowledge. By better understanding the origin and foundations of science, we can better understand their limits and, most importantly, how to push those limits. We can do so especially by addressing the evolved cognitive constraints and biases we face and improving the methods we use., I am also grateful for funding from the European Commission (Marie Curie programme, grant 745447) and the Ministry of Science and Innovation of the Government of Spain (Ramón y Cajal programme, grant RYC2020-029424-I). For a broader discussion of the foundations of science that applies other methods, see my forthcoming book: The Motor of Scientific Discovery., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Generalization Bias in Science

  • Peters, Uwe
  • Krauss, Alexander
  • Braganza, Oliver
Many scientists routinely generalize from study samples to larger populations. It is commonly assumed that this cognitive process of scientific induction is a voluntary inference in which researchers assess the generalizability of their data and then draw conclusions accordingly. We challenge this view and argue for a novel account. The account describes scientific induction as involving by default a generalization bias that operates automatically and frequently leads researchers to unintentionally generalize their findings without sufficient evidence. The result is unwarranted, overgeneralized conclusions. We support this account of scientific induction by integrating a range of disparate findings from across the cognitive sciences that have until now not been connected to research on the nature of scientific induction. The view that scientific induction involves by default a generalization bias calls for a revision of the current thinking about scientific induction and highlights an overlooked cause of the replication crisis in the sciences. Commonly proposed interventions to tackle scientific overgeneralizations that may feed into this crisis need to be supplemented with cognitive debiasing strategies against generalization bias to most effectively improve science., A.K. was funded by the Ministry of Science and Innovation of the Government of Spain(Grant RYC2020-029424-I)
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Redefining the scientific method: as the use of sophisticated scientific methods that extend our mind

  • Krauss, Alexander
Scientific, medical, and technological knowledge has transformed our world, but we still poorly understand the nature of scientific methodology. Science textbooks, science dictionaries, and science institutions often state that scientists follow, and should follow, the universal scientific method of testing hypotheses using observation and experimentation. Yet, scientific methodology has not been systematically analyzed using large-scale data and scientific methods themselves as it is viewed as not easily amenable to scientific study. Using data on all major discoveries across science including all Nobel Prize and major non-Nobel Prize discoveries, we can address the question of the extent to which "the scientific method" is actually applied in making science's groundbreaking research and whether we need to expand this central concept of science. This study reveals that 25% of all discoveries since 1900 did not apply the common scientific method (all three features)-with 6% of discoveries using no observation, 23% using no experimentation, and 17% not testing a hypothesis. Empirical evidence thus challenges the common view of the scientific method. Adhering to it as a guiding principle would constrain us in developing many new scientific ideas and breakthroughs. Instead, assessing all major discoveries, we identify here a general, common feature that the method of science can be reduced to: making all major discoveries has required using sophisticated methods and instruments of science. These include statistical methods, particle accelerators, and X-ray methods. Such methods extend our mind and generally make observing, experimenting, and testing hypotheses in science possible, doing so in new ways and ensure their replicability. This provides a new perspective to the scientific method-embedded in our sophisticated methods and instruments-and suggests that we need to reform and extend the way we view the scientific method and discovery process., This research was funded by the European Commission (grant 745447) and the Ministry of Science and Innovation of the Government of Spain (grant RYC2020-029424-I)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Science’s greatest discoverers: a shift towards greater interdisciplinarity, top universities and older age

  • Krauss, Alexander
What are the unique features and characteristics of the scientists who have made the greatest discoveries in science? To address this question, we assess all major scientific discoverers, defined as all nobel-prize and major non-nobel-prize discoverers, and their demographic, institutional and economic traits. What emerges is a general profile of the scientists who have driven over 750 of science’s greatest advances. We find that interdisciplinary scientists who completed two or more degrees in different academic fields by the time of discovery made about half—54%—of all nobel-prize discoveries and 42% of major non-nobel-prize discoveries over the same period; this enables greater interdisciplinary methodological training for making new scientific achievements. Science is also becoming increasingly elitist, with scientists at the top 25 ranked universities accounting for 30% of both all nobel-prize and non-nobel-prize discoveries. Scientists over the age of 50 made only 7% of all nobel-prize discoveries and 15% of non-nobel-prize discoveries and those over the age of 60 made only 1% and 3%, respectively. The gap in years between making nobel-prize discoveries and receiving the award is also increasing over time across scientific fields—illustrating that it is taking longer to recognise and select major breakthroughs. Overall, we find that those who make major discoveries are increasingly interdisciplinary, older and at top universities. We also assess here the role and distribution of factors like geographic location, gender, religious affiliation and country conditions of these leading scientists, and how these factors vary across time and scientific fields. The findings suggest that more discoveries could be made if science agencies and research institutions provide greater incentives for researchers to work against the common trend of narrow specialisation and instead foster interdisciplinary research that combines novel methods across fields., I received funding from the Ministry of Science and Innovation of the Government of Spain (grant RYC2020-029424-I). For a broader discussion of what drives scientific discoveries, see also my forthcoming book: The Motor of Scientific Discovery., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

How nobel-prize breakthroughs in economics emerge and the field's influential empirical methods

  • Krauss, Alexander
What drives groundbreaking research in economics? Nobel-prize-winning work has had an important impact on public policies, but we still do not understand well what drives such breakthroughs. We collect data on all nobel-prize discoveries in economics to address this question. We find that major advances in the field of economics are brought about by methodological innovation: by developing new and improved research methods. We find that developing for example econometrics in 1933, randomised controlled trials in 1948 and new game theory methods in 1950 were essential to opening the new fields of corporate finance, experimental economics and information economics, respectively. We identify the development of new methods as the main mechanism driving new discoveries and research fields. Fostering this general mechanism (generating novel methods) holds the potential to greatly increase the rate at which we make new breakthroughs and fields. We also show that many of the main methods of economics – such as randomised controlled trials, natural experiments, regression discontinuity, instrumental variables and other statistical methods – had been developed and used in other fields like public health, before economists adopted them. This shift towards more powerful empirical methods in the field has important implications on developing new and better methods and adopting them from related fields to make new advances more rapidly., This research received funding from the Ministry of Science and Innovation of the Government of Spain (grant RYC2020–029424-I), from the project ‘Institutional and Policy Design to Enhance Social Welfare’ (grant PID2021–126200NB-I00), the Generalitat de Catalunya (grant 2021-SGR-00416 ), and from Spain's State Research Agency (AEI), through the Severo Ochoa Programme for Centres of Excellence in R&D (Barcelona School of Economics CEX2019–000915-S)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Science of science: A multidisciplinary field studying science

  • Krauss, Alexander
Science and knowledge are studied by researchers across many disciplines, examining how they are developed, what their current boundaries are and how we can advance them. By integrating evidence across disparate disciplines, the holistic field of science of science can address these foundational questions. This field illustrates how science is shaped by many interconnected factors: the cognitive processes of scientists, the historical evolution of science, economic incentives, institutional influences, computational approaches, statistical, mathematical and instrumental foundations of scientific inference, scientometric measures, philosophical and ethical dimensions of scientific concepts, among other influences. Achieving a comprehensive overview of a multifaceted field like the science of science requires pulling together evidence from the many sub-fields studying science across the natural and social sciences and humanities. This enables developing an interdisciplinary perspective of scientific practice, a more holistic understanding of scientific processes and outcomes, and more nuanced perspectives to how scientific research is conducted, influenced and evolves. It enables leveraging the strengths of various disciplines to create a holistic view of the foundations of science. Different researchers study science from their own disciplinary perspective and use their own methods, and there is a large divide between quantitative and qualitative researchers as they commonly do not read or cite research using other methodological approaches. A broader, synthesizing paper employing a qualitative approach can however help provide a bridge between disciplines by pulling together aspects of science (economic, scientometric, psychological, philosophical etc.). Such an approach enables identifying, across the range of fields, the powerful role of our scientific methods and instruments in shaping most aspects of our knowledge and science, whereas economic, social and historical influences help shape what knowledge we pursue. A unifying theory is then outlined for science of science - the new-methods-drive-science theory., I am grateful for funding received by the European Commission (Marie Curie programme) and the Ministry of Science and Innovation of the Government of Spain (grant RYC2020-029424-I, and PID2021-126200NB-I00)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Debunking revolutionary paradigm shifts: evidence of cumulative scientific progress across science

  • Krauss, Alexander
How can scientific progress be conceived best? Does science mainly undergo revolutionary paradigm shifts? Or is the evolution of science mainly cumulative? Understanding whether science advances through cumulative evolution or through paradigm shifts can influence how we approach scientific research, education and policy. The most influential and cited account of science was put forth in Thomas Kuhn’s seminal book The structure of scientific revolutions. Kuhn argues that science does not advance cumulatively but goes through fundamental paradigm changes in the theories of a scientific field. There is no consensus yet on this core question of the nature and advancement of science that has since been debated across science. Examining over 750 major scientific discoveries (all Nobel Prize and major non-Nobel Prize discoveries), we systematically test this fundamental question about scientific progress here. We find that three key measures of scientific progress—major discoveries, methods and fields—each demonstrate that science evolves cumulatively. First, we show that no major scientific methods or instruments used across fields (such as statistical methods, X-ray methods or chromatography) have been completely abandoned, i.e. subject to paradigm shifts. Second, no major scientific fields (such as biomedicine, chemistry or computer science) have been completely abandoned. Rather, they have all continuously expanded over time, often over centuries, accumulating extensive bodies of knowledge. Third, scientific discoveries including theoretical discoveries are also predominately cumulative, with only 1% of over 750 major discoveries having been abandoned. The continuity of science is most compellingly evidenced by our methods and instruments, which enable the creation of discoveries and fields. We thus offer here a new perspective and answer to this classic question in science and the philosophy and history of science by utilizing methods from statistics and empirical sciences., This research was funded by the European Commission (Marie Curie programme) and the Ministry of Science and Innovation of the Government of Spain (grant RYC2020-029424-I)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Otros (Other)

Multi-Studies: A Novel Approach to Addressing Irreplicability in RCTs

  • Krauss, Alexander
© The Author(s), under exclusive license to Springer Nature, This chapter proposes a novel approach to addressing irreplicability encountered in randomised controlled trials (RCTs). Irreplicability affects, to different degrees, all fields of science. Some of the most common explanations for non-replicability are low sample size, low statistical power, p-hacking, publication bias and HARKing. Such issues face some studies but not others. Focusing on RCTs, this chapter highlights that, while it is well-recognised that many types of bias and other types of constraint generally face studies across the biomedical, behavioural and social sciences, they are not commonly discussed collectively. In response to this gap in the literature on RCTs, we shall consider a novel approach to addressing non-replicability. The chapter helps explain how a degree of inevitable variation between the outcomes of different studies is driven by a combined set of biases and constraints underlying the scientific process that relate to methods, people, contexts, and time points at which data are collected in studies. It outlines how causal evidence in such studies is inferred from estimated statistical results that are the outcome of multiple complex processes: sampling, randomising, blinding, controlling, delivering treatments, and so on. These processes involve many actors making many decisions at different steps when designing, implementing and analysing studies within a particular context and time period. Correspondingly, the chapter considers the influence of the combined set of constraints on the quality and replicability of RCTs, and how to better tackle this issue. This entails integrating comparability of groups, treatments, contexts, etc. within a single study—called here a multi-study. Implications are then drawn for practitioners., I am thankful for comments from Nancy Cartwright, John Worrall, Federica Russo, Carl Hoefer, Stephan Guettinger, Corinna Peters, Camille Lassale, Federica Malfatti and Saana Jukola. I received funding from the Marie Curie programme of the European Commission and the Ministry of Science and Innovation of the Government of Spain (grant RYC2020-029424-I)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

New scientific fields are triggered by powerful new methods

  • Krauss, Alexander
Scientific fields embody our greatest scientific advances, but we do not yet understand how we give rise to new fields. Explaining empirically and theoretically how we kick-start new fields has the potential to accelerate scientific progress. No comprehensive answer to this fundamental question yet exists. Here we systematically trace the origins of science’s major fields including over 350 fields spanning across science. We do this by analysing the methods and tools that enabled sparking the fields and link them to the broader conditions of the scientists who created the fields. This provides a unique opportunity to identify the common mechanism driving new fields. We find that fields consistently emerge by developing a new method or tool – from advanced telescopes to electrophoresis – as they enabled a completely new perspective to the world and without them, the fields would not have been possible. About a quarter of fields are the new method or tool themselves, such as laser physics, computer science, x-ray crystallography, and econometrics, forming entire disciplines around novel techniques. Our extraordinary development of new statistical techniques, x-ray devices, microscopes and spectrometers each made over ten new fields possible. The common link uniting these diverse fields is not specific theories, large teams, more funding or even serendipity – it is that each field relied on the same kind of powerful tool, used in remarkably different domains. The speed at which science expands is not random. The pace of opening new research domains is mainly determined by the pace at which we create new tools: particle detectors launched high-energy physics, microscopy techniques triggered neuroscience and randomised controlled trials kick-started experimental economics. This simple yet powerful principle – if we begin to deliberately develop transformative methods and tools – holds the key to enabling a tool revolution in science, changing the way we understand and the speed at which we make scientific progress. This methods-driven mechanism can provide a foundation for the field of Science of Science. It also points to the need for a new field targeted to the development of methods: Methodology of Science., I am thankful for comments from Nikolas Schöll, Corinna Peters, Milan Quentel, Uwe Peters, Joseph Emmens, Julia Hoefer Marti, Alfonso García Lapeña and Nebojsa Rodic. I received funding from the Ministry of Science and Innovation of the Government of Spain (grant RYC2020-029424-I, PID2021-126200NB-I00, and PID2024-158997NB-I00)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Discovery tools: How powerful new scientific methods and instruments emerge and catalyze innovation

  • Krauss, Alexander
Powerful new methods and tools drive scientific progress-but how do we actually make such innovations? No theory yet explains how we invent major tools across fields. To address this gap, we examine all nobel-prize-winning method discoveries-that enabled breakthrough findings not possible without them-and we trace science's most influential toolmakers across fields, from Ernst Ruska's electron microscope and Kary Mullis's PCR method to Ernest Lawrence's particle accelerator. Here, we lay out the critical pathways taken to create these groundbreaking tools. We introduce a taxonomy of science's methods and tools: a scientific table of discovery methods that is a map of underexplored and unexplored method opportunities. By mapping the method landscape, we reveal gaps and possibilities that help guide where methods can be adapted, recombined, or strategically developed to catalyze discovery. It can help identify and predict untapped combinations of tools-and where the next breakthroughs can come from. What if we no longer wait for new discovery tools to emerge by chance but begin deliberately prioritizing their development? How many big breakthroughs are we missing because we have not yet strategically focused on designing the needed tools? We also outline the need for establishing methods labs and hubs-as incubators of innovation-that catalyze tool creation., The research was funded by the Ministry of Science and Innovation of the Government of Spain (grants RYC2020- 029424-I and PID2024-158997NB-I00)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Predictable serendipity: how new tools turn serendipity into systematic breakthroughs

  • Krauss, Alexander
The emergence of scientific discoveries is often seen as the most extraordinary and puzzling feature of science. Many think that discoveries arise due to serendipity or chance—that they are unexplainable or unpredictable. Existing studies have not yet explained the extent and role of serendipity in science, because they generally only study a sample of discoveries that does not allow for generalising about science. To do so, this is the first study to assess science’s major discoveries, defined as all nobel-prize and major non-nobel-prize discoveries across fields and history. By examining over 750 major discoveries, we find that discoveries commonly labelled as serendipitous are actually sparked by using—for the first time—a powerful new tool that makes the unexpected observation possible that we are not even looking for: an improved microscope revealed cells, a discharge tube uncovered X-rays, gel diffusion revealed the Hepatitis B virus and a cutting-edge spectrograph detected the first planet outside our solar system. What seems like chance is often just using an entirely new lens—just as in each of these discoveries. In other words, we show that breakthroughs that seem serendipitous follow shortly after we create the new tool, with other researchers able to arrive at the same findings using these same discovery tools. While a serendipitous moment was reported in about one in six major discoveries, there is a pattern behind the surprise: using new tools to make the breakthrough finding commonly through exploratory research. This is important because it changes how we think about discovery: we can generally better understand individual serendipitous discoveries as tool-triggered discoveries that multiple researchers can make. Crucially, new method innovations provide explanatory power to help predict discoveries—whether labelled serendipitous or not. So here we reframe discovery: because new tools are what commonly unlock surprise, we can actually strategically design and accelerate discovery, shifting it from a more passive outcome of serendipity., Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This work is supported by the Ministry of Science and Innovation of the Government of Spain, Grant RYC2020-029424-I., Peer reviewed