-
Metrics and Mechanisms: Measuring the Unmeasurable in the Science of Science
Authors:
Lingfei Wu,
Aniket Kittur,
Hyejin Youn,
Staša Milojević,
Erin Leahey,
Stephen M. Fiore,
Yong Yeol Ahn
Abstract:
What science does, what science could do, and how to make science work? If we want to know the answers to these questions, we need to be able to uncover the mechanisms of science, going beyond metrics that are easily collectible and quantifiable. In this perspective piece, we link metrics to mechanisms by demonstrating how emerging metrics of science not only offer complementaries to existing ones…
▽ More
What science does, what science could do, and how to make science work? If we want to know the answers to these questions, we need to be able to uncover the mechanisms of science, going beyond metrics that are easily collectible and quantifiable. In this perspective piece, we link metrics to mechanisms by demonstrating how emerging metrics of science not only offer complementaries to existing ones, but also shed light on the hidden structure and mechanisms of science. Based on fundamental properties of science, we classify existing theories and findings into: hot and cold science referring to attention shift between scientific fields, fast and slow science reflecting productivity of scientists and teams, soft and hard science revealing reproducibility of scientific research. We suggest that interest about mechanisms of science since Derek J. de Solla Price, Robert K. Merton, Eugene Garfield, and many others complement the zeitgeist in pursuing new, complex metrics without understanding the underlying processes. We propose that understanding and modeling the mechanisms of science condition effective development and application of metrics.
△ Less
Submitted 9 April, 2022; v1 submitted 14 November, 2021;
originally announced November 2021.
-
Unsupervised embedding of trajectories captures the latent structure of scientific migration
Authors:
Dakota Murray,
Jisung Yoon,
Sadamori Kojaku,
Rodrigo Costas,
Woo-Sung Jung,
Staša Milojević,
Yong-Yeol Ahn
Abstract:
Human migration and mobility drives major societal phenomena including epidemics, economies, innovation, and the diffusion of ideas. Although human mobility and migration have been heavily constrained by geographic distance throughout the history, advances and globalization are making other factors such as language and culture increasingly more important. Advances in neural embedding models, origi…
▽ More
Human migration and mobility drives major societal phenomena including epidemics, economies, innovation, and the diffusion of ideas. Although human mobility and migration have been heavily constrained by geographic distance throughout the history, advances and globalization are making other factors such as language and culture increasingly more important. Advances in neural embedding models, originally designed for natural language, provide an opportunity to tame this complexity and open new avenues for the study of migration. Here, we demonstrate the ability of the model word2vec to encode nuanced relationships between discrete locations from migration trajectories, producing an accurate, dense, continuous, and meaningful vector-space representation. The resulting representation provides a functional distance between locations, as well as a digital double that can be distributed, re-used, and itself interrogated to understand the many dimensions of migration. We show that the unique power of word2vec to encode migration patterns stems from its mathematical equivalence with the gravity model of mobility. Focusing on the case of scientific migration, we apply word2vec to a database of three million migration trajectories of scientists derived from the affiliations listed on their publication records. Using techniques that leverage its semantic structure, we demonstrate that embeddings can learn the rich structure that underpins scientific migration, such as cultural, linguistic, and prestige relationships at multiple levels of granularity. Our results provide a theoretical foundation and methodological framework for using neural embeddings to represent and understand migration both within and beyond science.
△ Less
Submitted 17 November, 2023; v1 submitted 4 December, 2020;
originally announced December 2020.
-
Recency predicts bursts in the evolution of author citations
Authors:
Filipi Nascimento Silva,
Aditya Tandon,
Diego Raphael Amancio,
Alessandro Flammini,
Filippo Menczer,
Staša Milojević,
Santo Fortunato
Abstract:
The citations process for scientific papers has been studied extensively. But while the citations accrued by authors are the sum of the citations of their papers, translating the dynamics of citation accumulation from the paper to the author level is not trivial. Here we conduct a systematic study of the evolution of author citations, and in particular their bursty dynamics. We find empirical evid…
▽ More
The citations process for scientific papers has been studied extensively. But while the citations accrued by authors are the sum of the citations of their papers, translating the dynamics of citation accumulation from the paper to the author level is not trivial. Here we conduct a systematic study of the evolution of author citations, and in particular their bursty dynamics. We find empirical evidence of a correlation between the number of citations most recently accrued by an author and the number of citations they receive in the future. Using a simple model where the probability for an author to receive new citations depends only on the number of citations collected in the previous 12-24 months, we are able to reproduce both the citation and burst size distributions of authors across multiple decades.
△ Less
Submitted 26 November, 2019;
originally announced November 2019.
-
Network dynamics of innovation processes
Authors:
Iacopo Iacopini,
Staša Milojević,
Vito Latora
Abstract:
We introduce a model for the emergence of innovations, in which cognitive processes are described as random walks on the network of links among ideas or concepts, and an innovation corresponds to the first visit of a node. The transition matrix of the random walk depends on the network weights, while in turn the weight of an edge is reinforced by the passage of a walker. The presence of the networ…
▽ More
We introduce a model for the emergence of innovations, in which cognitive processes are described as random walks on the network of links among ideas or concepts, and an innovation corresponds to the first visit of a node. The transition matrix of the random walk depends on the network weights, while in turn the weight of an edge is reinforced by the passage of a walker. The presence of the network naturally accounts for the mechanism of the adjacent possible, and the model reproduces both the rate at which novelties emerge and the correlations among them observed empirically. We show this by using synthetic networks and by studying real data sets on the growth of knowledge in different scientific disciplines. Edge-reinforced random walks on complex topologies offer a new modeling framework for the dynamics of correlated novelties and are another example of coevolution of processes and networks.
△ Less
Submitted 24 January, 2018; v1 submitted 13 July, 2017;
originally announced July 2017.
-
Citation success index - An intuitive pair-wise journal comparison metric
Authors:
Staša Milojević,
Filippo Radicchi,
Judit Bar-Ilan
Abstract:
In this paper we present "citation success index", a metric for comparing the citation capacity of pairs of journals. Citation success index is the probability that a random paper in one journal has more citations than a random paper in another journal (50% means the two journals do equally well). Unlike the journal impact factor (IF), the citation success index depends on the broadness and the sh…
▽ More
In this paper we present "citation success index", a metric for comparing the citation capacity of pairs of journals. Citation success index is the probability that a random paper in one journal has more citations than a random paper in another journal (50% means the two journals do equally well). Unlike the journal impact factor (IF), the citation success index depends on the broadness and the shape of citation distributions. Also, it is insensitive to sporadic highly-cited papers that skew the IF. Nevertheless, we show, based on 16,000 journals containing ~2.4 million articles, that the citation success index is a relatively tight function of the ratio of IFs of journals being compared, due to the fact that journals with same IF have quite similar citation distributions. The citation success index grows slowly as a function of IF ratio. It is substantial (>90%) only when the ratio of IFs exceeds ~6, whereas a factor of two difference in IF values translates into a modest advantage for the journal with higher IF (index of ~70%). We facilitate the wider adoption of this metric by providing an online calculator that takes as input parameters only the IFs of the pair of journals.
△ Less
Submitted 21 December, 2016; v1 submitted 11 July, 2016;
originally announced July 2016.
-
Quantifying the Cognitive Extent of Science
Authors:
Staša Milojević
Abstract:
While the modern science is characterized by an exponential growth in scientific literature, the increase in publication volume clearly does not reflect the expansion of the cognitive boundaries of science. Nevertheless, most of the metrics for assessing the vitality of science or for making funding and policy decisions are based on productivity. Similarly, the increasing level of knowledge produc…
▽ More
While the modern science is characterized by an exponential growth in scientific literature, the increase in publication volume clearly does not reflect the expansion of the cognitive boundaries of science. Nevertheless, most of the metrics for assessing the vitality of science or for making funding and policy decisions are based on productivity. Similarly, the increasing level of knowledge production by large science teams, whose results often enjoy greater visibility, does not necessarily mean that "big science" leads to cognitive expansion. Here we present a novel, big-data method to quantify the extents of cognitive domains of different bodies of scientific literature independently from publication volume, and apply it to 20 million articles published over 60-130 years in physics, astronomy, and biomedicine. The method is based on the lexical diversity of titles of fixed quotas of research articles. Owing to large size of quotas, the method overcomes the inherent stochasticity of article titles to achieve <1% precision. We show that the periods of cognitive growth do not necessarily coincide with the trends in publication volume. Furthermore, we show that the articles produced by larger teams cover significantly smaller cognitive territory than (the same quota of) articles from smaller teams. Our findings provide a new perspective on the role of small teams and individual researchers in expanding the cognitive boundaries of science. The proposed method of quantifying the extent of the cognitive territory can also be applied to study many other aspects of "science of science."
△ Less
Submitted 3 November, 2015; v1 submitted 30 October, 2015;
originally announced November 2015.
-
Principles of scientific research team formation and evolution
Authors:
Staša Milojević
Abstract:
Research teams are the fundamental social unit of science, and yet there is currently no model that describes their basic property: size. In most fields teams have grown significantly in recent decades. We show that this is partly due to the change in the character of team-size distribution. We explain these changes with a comprehensive yet straightforward model of how teams of different sizes eme…
▽ More
Research teams are the fundamental social unit of science, and yet there is currently no model that describes their basic property: size. In most fields teams have grown significantly in recent decades. We show that this is partly due to the change in the character of team-size distribution. We explain these changes with a comprehensive yet straightforward model of how teams of different sizes emerge and grow. This model accurately reproduces the evolution of empirical team-size distribution over the period of 50 years. The modeling reveals that there are two modes of knowledge production. The first and more fundamental mode employs relatively small, core teams. Core teams form by a Poisson process and produce a Poisson distribution of team sizes in which larger teams are exceedingly rare. The second mode employs extended teams, which started as core teams, but subsequently accumulated new members proportional to the past productivity of their members. Given time, this mode gives rise to a power-law tail of large teams (10-1000 members), which features in many fields today. Based on this model we construct an analytical functional form that allows the contribution of different modes of authorship to be determined directly from the data and is applicable to any field. The model also offers a solid foundation for studying other social aspects of science, such as productivity and collaboration.
△ Less
Submitted 11 March, 2014;
originally announced March 2014.
-
Citation content analysis (cca): A framework for syntactic and semantic analysis of citation content
Authors:
Guo Zhang,
Ying Ding,
Staša Milojević
Abstract:
This paper proposes a new framework for Citation Content Analysis (CCA), for syntactic and semantic analysis of citation content that can be used to better analyze the rich sociocultural context of research behavior. The framework could be considered the next generation of citation analysis. This paper briefly reviews the history and features of content analysis in traditional social sciences, and…
▽ More
This paper proposes a new framework for Citation Content Analysis (CCA), for syntactic and semantic analysis of citation content that can be used to better analyze the rich sociocultural context of research behavior. The framework could be considered the next generation of citation analysis. This paper briefly reviews the history and features of content analysis in traditional social sciences, and its previous application in Library and Information Science. Based on critical discussion of the theoretical necessity of a new method as well as the limits of citation analysis, the nature and purposes of CCA are discussed, and potential procedures to conduct CCA, including principles to identify the reference scope, a two-dimensional (citing and cited) and two-modular (syntactic and semantic modules) codebook, are provided and described. Future works and implications are also suggested.
△ Less
Submitted 27 November, 2012;
originally announced November 2012.
-
How are academic age, productivity and collaboration related to citing behavior of researchers?
Authors:
Staša Milojević
Abstract:
References are an essential component of research articles and therefore of scientific communication. In this study we investigate referencing (citing) behavior in five diverse fields (astronomy, mathematics, robotics, ecology and economics) based on 213,756 core journal articles. At the macro level we find: (a) a steady increase in the number of references per article over the period studied (50…
▽ More
References are an essential component of research articles and therefore of scientific communication. In this study we investigate referencing (citing) behavior in five diverse fields (astronomy, mathematics, robotics, ecology and economics) based on 213,756 core journal articles. At the macro level we find: (a) a steady increase in the number of references per article over the period studied (50 years), which in some fields is due to a higher rate of usage, while in others reflects longer articles and (b) an increase in all fields in the fraction of older, foundational references since the 1980s, with no obvious change in citing patterns associated with the introduction of the Internet. At the meso level we explore current (2006-2010) referencing behavior of different categories of authors (21,562 total) within each field, based on their academic age, productivity and collaborative practices. Contrary to some previous findings and expectations we find that senior researchers use references at the same rate as their junior colleagues, with similar rates of re-citation (use of same references in multiple papers). High Modified Price Index (MPI, which measures the speed of the research front more accurately than the traditional Price Index) of senior authors indicates that their research has the similar cutting-edge aspect as that of their younger colleagues. In all fields both the productive researchers and especially those who collaborate more use a significantly lower fraction of foundational references and have much higher MPI and lower re-citation rates, i.e., they are the ones pushing the research front regardless of researcher age. This paper introduces improved bibliometric methods to measure the speed of the research front, disambiguate lead authors in co-authored papers and decouple measures of productivity and collaboration.
△ Less
Submitted 21 November, 2012; v1 submitted 13 October, 2012;
originally announced October 2012.
-
Social Dynamics of Science
Authors:
Xiaoling Sun,
Jasleen Kaur,
Staša Milojević,
Alessandro Flammini,
Filippo Menczer
Abstract:
The birth and decline of disciplines are critical to science and society. However, no quantitative model to date allows us to validate competing theories of whether the emergence of scientific disciplines drives or follows the formation of social communities of scholars. Here we propose an agent-based model based on a \emph{social dynamics of science,} in which the evolution of disciplines is guid…
▽ More
The birth and decline of disciplines are critical to science and society. However, no quantitative model to date allows us to validate competing theories of whether the emergence of scientific disciplines drives or follows the formation of social communities of scholars. Here we propose an agent-based model based on a \emph{social dynamics of science,} in which the evolution of disciplines is guided mainly by the social interactions among scientists. We find that such a social theory can account for a number of stylized facts about the relationships between disciplines, authors, and publications. These results provide strong quantitative support for the key role of social interactions in shaping the dynamics of science. A "science of science" must gauge the role of exogenous events, such as scientific discoveries and technological advances, against this purely social baseline.
△ Less
Submitted 21 September, 2012;
originally announced September 2012.
-
Multidisciplinary Cognitive Content of Nanoscience and Nanotechnology
Authors:
Staša Milojević
Abstract:
This article examines the cognitive evolution and disciplinary diversity of nanotechnology as expressed through the terminology used in titles of nano journal articles. The analysis is based on the NanoBank bibliographic database of 287,106 nano articles published between 1981 and 2004. We perform multifaceted analyses of title words, focusing on 100 most frequent terms. Hierarchical clustering of…
▽ More
This article examines the cognitive evolution and disciplinary diversity of nanotechnology as expressed through the terminology used in titles of nano journal articles. The analysis is based on the NanoBank bibliographic database of 287,106 nano articles published between 1981 and 2004. We perform multifaceted analyses of title words, focusing on 100 most frequent terms. Hierarchical clustering of title terms reveals three distinct time periods of cognitive development of nano research: formative (1981-1990), early (1991-1998), and current (after 1998). Early period is characterized by the introduction of thin film deposition techniques, while the current period is characterized by the increased focus on carbon nanotube and nanoparticle research. We introduce a method to identify disciplinary components of nanotechnology. It shows that the nano research is being carried out in a number of diverse parent disciplines. Currently only 5% of articles are published in dedicated nano-only journals. We find that some 85% of nano research today is multidisciplinary. Hierarchical clustering of disciplinary components reveals that the cognitive content of current nanoscience can be divided into nine clusters. Some clusters account for a large fraction of nano research and are identified with such parent disciplines as the condensed matter and applied physics, materials science, and analytical chemistry. Other clusters represent much smaller parts of nano research, but are as cognitively distinct. In the decreasing order of size, these fields are: polymer science, biotechnology, general chemistry, surface science, and pharmacology. Cognitive content of research published in nano-only journals is closest to nano research published in condensed matter and applied physics journals.
△ Less
Submitted 31 March, 2012;
originally announced April 2012.
-
Power-law Distributions in Information Science - Making the Case for Logarithmic Binning
Authors:
Staša Milojević
Abstract:
We suggest partial logarithmic binning as the method of choice for uncovering the nature of many distributions encountered in information science (IS). Logarithmic binning retrieves information and trends "not visible" in noisy power-law tails. We also argue that obtaining the exponent from logarithmically binned data using a simple least square method is in some cases warranted in addition to met…
▽ More
We suggest partial logarithmic binning as the method of choice for uncovering the nature of many distributions encountered in information science (IS). Logarithmic binning retrieves information and trends "not visible" in noisy power-law tails. We also argue that obtaining the exponent from logarithmically binned data using a simple least square method is in some cases warranted in addition to methods such as the maximum likelihood. We also show why often used cumulative distributions can make it difficult to distinguish noise from genuine features, and make it difficult to obtain an accurate power-law exponent of the underlying distribution. The treatment is non-technical, aimed at IS researchers with little or no background in mathematics.
△ Less
Submitted 5 November, 2010;
originally announced November 2010.
-
Modes of Collaboration in Modern Science - Beyond Power Laws and Preferential Attachment
Authors:
Staša Milojević
Abstract:
The goal of the study is to determine the underlying processes leading to the observed collaborator distribution in modern scientific fields, with special attention to non-power law behavior. Nanoscience is used as a case study of a modern interdisciplinary field, and its coauthorship network for 2000-04 period is constructed from NanoBank database. We find three collaboration modes that correspon…
▽ More
The goal of the study is to determine the underlying processes leading to the observed collaborator distribution in modern scientific fields, with special attention to non-power law behavior. Nanoscience is used as a case study of a modern interdisciplinary field, and its coauthorship network for 2000-04 period is constructed from NanoBank database. We find three collaboration modes that correspond to three distinct ranges in the distribution of collaborators: (1) for authors with fewer than 20 collaborators (the majority) preferential attachment does not hold and they form a log-normal "hook" instead of a power law, (2) authors with more than 20 collaborators benefit from preferential attachment and form a power law tail, and (3) authors with between 250 and 800 collaborators are more frequent than expected because of the hyperauthorship practices in certain subfields.
△ Less
Submitted 28 April, 2010;
originally announced April 2010.
