Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 9
Encontrada(s) 1 página(s)

e-Archivo. Repositorio Institucional de la Universidad Carlos III de Madrid
Publication Artículo preliminar (WorkingPaper)

The citation merit of scientific publications

  • Crespo, Juan A.
  • Ortuño, Ignacio
  • Ruiz-Castillo, Javier
We propose a new method to assess the merit of any set of scientific papers in a given field based on the citations they receive. Given a citation indicator, such as the mean citation or the h-index, we identify the merit of a given set of n articles with the probability that a randomly drawn sample of an articles from a reference set of articles in that field presents a lower citation index. The method allows for comparisons between research units of different sizes and fields. Using a dataset acquired from Thomson Scientific that contains the articles published in the periodical literature in the period 1998-2007, we show that the novel approach yields rankings of research units different from those obtained by a direct application of the mean citation or the h-index., Ruiz-Castillo acknowledges financial help from the Spanish MEC through grant SEJ2007-67436. Crespo and Ortuño-Ortín also acknowledge financial help from the Spanish MEC through grant ECO2010-19596. This paper is produced as part of the project Science, Innovation, Firms and markets in a Globalised World (SCIFI-GLOW), a Collaborative Project funded by the European Commission's Seventh Research Framework Programme, Contract number SSH7-CT-2008-217436.
e-Archivo. Repositorio Institucional de la Universidad Carlos III de Madrid
Publication Artículo preliminar (WorkingPaper)

Sub-field normalization in the multiplicative case : high- and low- impact citation indicators

  • Herranz, Neus
  • Ruiz-Castillo, Javier
This paper uses high- and low-impact citation indicators for the evaluation of the citation performance of research units at different aggregate levels. To solve the problem of the assignment of individual articles to multiple sub-fields, it follows a multiplicative strategy according to which each paper is wholly counted as many times as necessary in the several categories to which it is assigned at each aggregation level. To control for wide differences in citation practices at the lowest level of aggregation, we apply a novel subfield normalization procedure in the multiplicative case. The methodology is applied to a partition of the world into three geographical areas: the U.S., the European Union (EU), and the Rest of the World. The main findings are the following two. (i) Although normalization does not systematically bias the results against any area, it reduces the U.S./EU highimpact gap in the all-sciences case by a non-negligible 14.4%. (ii) The dominance of the U.S. over the EU in the basic and applied research published in the periodical literature is almost universal at all aggregation levels. From the high-impact perspective, for example, the U.S. is ahead of the EU in 77 out of 80 disciplines, and all of 20 fields. For all sciences as a whole, the U.S. high-impact indicator is 61% greater than that of the EU., The authors acknowledge financial support from the Santander Universities Global Division of Banco Santander. Ruiz-Castillo also acknowledges financial help from the Spanish MEC through grant SEJ2007- 67436. This paper is part of the SCIFI-GLOW Collaborative Project supported by the European Commission’s Seventh Research Framework Programme, Contract number SSH7-CT-2008-217436, European Community's Seventh Framework Program
e-Archivo. Repositorio Institucional de la Universidad Carlos III de Madrid
Publication Artículo preliminar (WorkingPaper)

The measurement of scientific excellence around the world

  • Albarrán, Pedro
  • Ruiz-Castillo, Javier
This paper reviews scientific excellence in 38 countries and eight geographical areas using two sets of novel indicators of citation impact: a family of high-impact indicators imported from the poverty literature in Economics, and a set of indicators within the percentile rank approach. Among the main findings with a dataset of about 4.4 million from Thomson Scientific we emphasize the following three. (i) The proportion of articles of a research unit in the set formed by the 10% of the most cited papers in the world, and two important percentile rank indicators bring no novelty relative to a traditional average-based indicator. (ii) A high-impact indicator very sensitive to citation inequality is seen to be useful to detect success at a local level, but not for a global ranking that includes small research units. (iii) A monotonic high-impact indicator sensitive to any increase in citations is used to rank the partition of the world into 46 units in the 22 broad fields distinguished by Thomson Scientific, as well as the all-sciences case., The authors acknowledge financial support by Santander Universities Global Division of Banco Santander. Albarrán acknowledges additional financial support from the Spanish MEC through grants ECO2009-11165 y ECO2011-29751, and Ruiz-Castillo through grant SEJ2007-67436. This paper is produced as part of the project Science, Innovation, Firms and markets in a Globalised World (SCIFIGLOW), a Collaborative Project funded by the European Commission's Seventh Research Framework Programme, Contract number SSH7-CT-2008-217436.
e-Archivo. Repositorio Institucional de la Universidad Carlos III de Madrid
Publication Artículo preliminar (WorkingPaper)

Sub-field normalization in the multiplicative case : average-based citation indicators

  • Herranz, Neus
  • Ruiz-Castillo, Javier
This paper investigates the citation impact of three large geographical areas –the U.S., the European Union (EU), and the rest of the world (RW)– at different aggregation levels. The difficulty is that 42% of the 3.6 million articles in our Thomson Scientific dataset are assigned to several sub-fields among a set of 219 Web of Science categories. We follow a multiplicative approach in which every article is wholly counted as many times as it appears at each aggregation level. We compute the crown indicator and the Mean Normalized Citation Score (MNCS) using for the first time sub-field normalization procedures for the multiplicative case. We also compute a third indicator that does not correct for differences in citation practices across sub-fields. It is found that: (1) No geographical area is systematically favored (or penalized) by any of the two normalized indicators. (2) According to the MNCS, only in six out of 80 disciplines –but in none of 20 fields– is the EU ahead of the U.S. In contrast, the normalized U.S./EU gap is greater than 20% in 44 disciplines, 13 fields, and for all sciences as a whole. The dominance of the EU over the RW is even greater. (3) The U.S. appears to devote relatively more –and the RW less– publication effort to subfields with a high mean citation rate, which explains why the U.S./EU and EU/RW gaps for all sciences as a whole increase by 4.5 and 5.6 percentage points in the un-normalized case. The results with a fractional approach are very similar indeed, Financial support from the Santander Universities Global Division of Banco Santander. Ruiz-Castillo also acknowledges financial help from the Spanish MEC through grant SEJ2007-67436, European Community's Seventh Framework Program
e-Archivo. Repositorio Institucional de la Universidad Carlos III de Madrid
Publication Artículo preliminar (WorkingPaper)

High - and low-impact citation measures: empirical applications

  • Albarrán, Pedro
  • Ortuño, Ignacio
  • Ruiz-Castillo, Javier
This paper contains the first empirical applications of a novel methodology for comparing the citation distributions of research units working in the same homogeneous field. The paper considers a situation in which the world citation distribution in 22 scientific fields is partitioned into three geographical areas: the U.S., the European Union (EU), and the rest of the world (RW). Given a critical citation level (CCL), we suggest using two real valued indicators to describe the shape of each area’s distribution: a high- and a low-impact measure defined over the set of articles with citations below or above the CCL. It is found that, when the CCL is fixed at the 80th percentile of the world citation distribution, the U.S. performs dramatically better than the EU and the RW according to both indicators in all scientific fields. This superiority generally increases as we move from the incidence to the intensity and the citation inequality aspects of the phenomena in question. Surprisingly, changes observed when the CCL is increased from the 80th to the 95th percentile are of a relatively small order of magnitude. Finally, it is found that international co-authorship increases the high-impact and reduces the low-impact level in the three geographical areas. This is especially the case for the EU and the RW when they cooperate with the U.S., Albarrán and Ruiz-Castillo acknowledge financial help from the Spanish MEC, the first through grants SEJ2007-63098 and SEJ2006-05710, and the second through grant SEJ2007-67436. Finally, this paper is part of the SCIFI-GLOW Collaborative Project supported by the European Commission's Seventh Research Framework Programme, Contract number SSH7-CT-2008-217436.
e-Archivo. Repositorio Institucional de la Universidad Carlos III de Madrid
Publication Artículo preliminar (WorkingPaper)

The end of the European Paradox

  • Herranz, Neus
  • Ruiz-Castillo, Javier
This paper evaluates the European Paradox according to which Europe plays a leading world role in terms of scientific excellence, measured in terms of the number of publications, but lacks the entrepreneurial capacity of the U.S. to transform this excellent performance into innovation, growth, and jobs. Citation distributions for the U.S., the European Union (EU), and the rest of the world are evaluated using a pair of high- and low-impact indicators, as well as the mean citation rate. The dataset consists of 3.6 million articles published in 1998-2002 with a common five-year citation window. The analysis is carried at a low aggregation level: the 219 sub-fields identified with the Web of Science categories distinguished by Thomson Scientific. The problems posed by international co-authorship and the multiple assignments of articles to sub-fields are solved following a multiplicative strategy. In the first place, we find that, although the EU has more publications than the U.S. in 113 out of 219 sub-fields, the U.S. is ahead of the EU in 189 and 163 sub-fields in terms of the high- and low-impact indicators. In the second place, we verify that using the high-impact indicator the U.S./EU gap is usually greater than when using the mean citation rate., The authors acknowledge financial support from the Santander Universities Global Division of Banco Santander. Ruiz-Castillo also acknowledges financial help from the Spanish MEC through grant SEJ2007-67436. This paper is part of the SCIFI-GLOW Collaborative Project supported by the European Commission’s Seventh Research Framework Programme, Contract number SSH7-CT-2008-217436.
e-Archivo. Repositorio Institucional de la Universidad Carlos III de Madrid
Publication Artículo preliminar (WorkingPaper)

The evaluation of citation distributions

  • Ruiz-Castillo, Javier
This paper reviews a number of recent contributions that demonstrate that a blend of welfare economics and statistical analysis is useful in the evaluation of the citations received by scientific papers in the periodical literature. The paper begins by clarifying the role of citation analysis in the evaluation of research. Next, a summary of results about the citation distributions’ basic features at different aggregation levels is offered. These results indicate that citation distributions share the same broad shape, are highly skewed, and are often crowned by a power law. In light of this evidence, a novel methodology for the evaluation of research units is illustrated by comparing the high- and low-citation impact achieved by the U.S., the European Union, and the rest of the world in 22 scientific fields. However, contrary to recent claims, it is shown that mean normalization at the sub-field level does not lead to a universal distribution. Nevertheless, among other topics subject to ongoing research, it appears that this lack of universality does not preclude sensible normalization procedures to compare the citation impact of articles in different scientific fields., The author acknowledges financial support from Santander Universities Global Division of Banco Santander, as well as from the Spanish MEC through grant SEJ2007-67436. This paper is part of the SCIFI-GLOW Collaborative Project supported by the European Commission’s Seventh Research Framework Programme, Contract no. SSH7-CT-2008-217436.
e-Archivo. Repositorio Institucional de la Universidad Carlos III de Madrid
Publication Artículo preliminar (WorkingPaper)

Average-based versus high-and low-impact indicators for the evaluation of scientific distributions

  • Albarrán, Pedro
  • Ortuño, Ignacio
  • Ruiz-Castillo, Javier
Albarran et al. (2011a) introduced a novel methodology for the evaluation of citation distributions consisting of a pair of high- and a low-impact measures defined over the set of articles with citations below or above a critical citation level CCL. Albarran et al. (2011b) presented the first empirical applications to a situation in which the world citation distribution in 22 scientific fields is partitioned into three geographical areas: the U.S., the European Union, and the rest of the world. In this paper, we compare our results with those obtained with average-based indicators. For reasonable CCLs, such as the 80th percentile of the world citation distribution in each field, the cardinal differences between the results obtained with our high-impact index and the mean citation rate are of a large order of magnitude. When, in addition, the percentage in the top 5% of most cited articles or the percentage of uncited articles are used, there are still important quantitative differences with respect to the high- and low-impact indicators advocated in our approach when the CCL is fixed at the 80th or the 95th percentile., Albarrán y Ruiz-Castillo acknowledge financial help from the Spanish MEC, the first though grants SEJ2007-63098 and SEJ2006-05710, and the second through grant SEJ2007-67436. This paper is part of the SCIFI-GLOW Collaborative Project supported by the European Commission's Seventh Research Framework Programme, Contract number SSH7-CT-2008-217436.
e-Archivo. Repositorio Institucional de la Universidad Carlos III de Madrid
Publication Artículo preliminar (WorkingPaper)

The measurement of low- and high-impact in citation distributions : technical results

  • Albarrán, Pedro
  • Ortuño, Ignacio
  • Ruiz-Castillo, Javier
This paper introduces a novel methodology for comparing the citation distributions of research units working in the same homogeneous field. Given a critical citation level (CCL), we suggest using two real valued indicators to describe the shape of any distribution: a high-impact and a low-impact measure defined over the set of articles with citations above or below the CCL. The key to this methodology is the identification of a citation distribution with an income distribution. Once this step is taken, it is easy to realize that the measurement of low-impact coincides with the measurement of economic poverty. In turn, it is equally natural to identify the measurement of high-impact with the measurement of a certain notion of economic affluence. On the other hand, it is seen that the ranking of citation distributions according to a family of low-impact measures is essentially characterized by a number of desirable axioms. Appropriately redefined, these same axioms lead to the selection of an equally convenient class of decomposable high-impact measures. These two families are shown to satisfy other interesting properties that make them potentially useful in empirical applications, including the comparison of research units working in different fields., Albarrán and Ruiz-Castillo acknowledge financial support from the Spanish MEC, the former through grants SEJ2007-63098 and SEJ2006-05710, and the latter through grant SEJ2007-67436. This paper is part of the SCIFI-GLOW Collaborative Project supported by the European Commission's Seventh Research Framework Programme, Contract number SSH7-CT-2008-217436.