Every late June, academic publishing has its version of a holiday: the previous year's journal impact factors are released. Most journals can expect good news, because most impact factors rise. That ritual is a useful window into what the number measures, what it does not, and why citation inflation has become a subject in its own right.
Eugene Garfield later recalled in JAMA that the idea behind using citations for evaluation appeared in Science in 1955. That line of thinking led to the first Science Citation Index in 1961. Garfield and Irving H. Sher then developed the journal impact factor at the Institute for Scientific Information. ISI was later acquired by Thomson Reuters, and the Journal Citation Reports were sold as a subscription product. In 2016, Onex and Baring Private Equity Asia bought the business and created Clarivate, which now publishes the reports. The impact factor familiar today is, in theory, a Clarivate product.
The calculation itself is not complicated. A journal's 2021 impact factor is the number of citations in 2021 to items it published in 2019 and 2020, divided by the number of citable items—peer-reviewed research articles—published in those two years. If a journal published 100 such articles in 2019 and 2020 and those articles received 500 citations in 2021, its impact factor is 5. Because the formula has long been public, many journals can estimate their own figure by the end of the year even before JCR appears. The main inputs come from paid citation databases such as Web of Science and Scopus, though open-science pressure has made citation cross-checking more available at mainstream journals, so estimates usually land close to the official number.
It is worth remembering that the impact factor does not cover every academic journal. JCR currently includes roughly 20,000 journals. SCIE covers about 9,000. Scopus lists 43,000 titles, of which around 27,000 are active. Scimago also covers about 27,000 journals, and the older Chemical Abstracts database covers around 8,000. The total population of academic journals is probably in the neighborhood of 30,000. There are omissions, but because most papers receive very few citations, the practical distortion is limited.
The impact factor is a journal-level metric. Using it to judge people—saying that someone published in a journal with an impact factor of X—is a reversal of means and ends, rather like judging students by the prestige of their university. A new graduate from a famous school may get interviews, but someone still advertising the alma mater fifteen or twenty years later is probably relying on the brand because their own record cannot do the work. Hiring experienced people differs from campus recruiting: it depends more on demonstrated performance than on a school-name floor.
The same logic applies in research. Newcomers are unknown, so a journal's impact factor functions as a kind of credit advance. A group with a long record in a field has its own reputation; its work will be read wherever it appears. A 2009 study of papers in ChemPhysChem made the distribution vivid: citations within the journal were not Gaussian but closer to a power law, with a small number of highly cited papers dominating the impact factor.
A 2016 book reported that only about 30% of papers reach or exceed the impact factor of the journal that published them. For most articles, then, the journal impact factor is an overestimate. At the other end, highly cited papers have an incentive to advertise their own citation counts, because the journal figure underestimates them. Someone who keeps boasting about a high-impact paper is often someone whose citations are dragging the journal average down. Unfortunately, that describes most people; few have achieved impact-factor freedom.
Researchers often use impact factors to evaluate work or people outside their own specialty. That is understandable: academia is large, and outsiders need some quantitative shortcut. Online discussion of impact factors is dominated by graduate students, journal editors, and editorial boards. People who spend their careers in research pay less attention, though they still care. New journals and preprint servers have shifted their focus toward their own citation counts and influence, which help more directly with grants. Job candidates may still advertise the impact factor of the journal where their paper appeared; others advertise cover articles or highly cited papers. Serious evaluators look for work that differs from the candidate's pre-independent research direction, because that says more about future potential. Otherwise a department may simply be hiring a remote branch of a large laboratory—though sometimes that kind of team building matters too.
The impact factor has attracted controversy since it appeared. One problem is the misuse of a journal metric to judge individuals. Another is the numerator and denominator. The denominator counts only research articles, while the numerator can count citations from anything. Journals therefore have an incentive to add editorials, perspectives, letters, and similar items. Those are usually not peer reviewed, so they do not enter the denominator, but they can cite papers or highlight research and thereby add citations to the numerator. Review journals present a related case: reviews tend to collect many citations, so a new journal may launch with reviews by senior researchers and quickly build a high starting point.
Citation half-life is another issue. The impact factor uses a two-year window. In fast-moving fields, a paper older than two years may have little reading value, but in many fields, especially basic disciplines, the bulk of citations arrive later. Research from the Nature Index site found that biomedical papers peak in citations at about two years, while social science papers have roughly flat annual citation rates over their first ten years.
Using impact factors to compare fields is therefore problematic. A paper from a high-impact-factor field moving into a low-impact-factor field under the banner of interdisciplinarity, or using that disparity in grant applications, is not a fair comparison. Of course, that kind of export competition can also create pressure on traditional fields and push them to develop.
Self-citation is a further point of contention. A 2007 study described a Matthew effect in journal citations: strong journals are more willing to cite their own papers, and other journals cite strong journals, so authority and high impact factors reinforce each other. In 2016, overall journal self-citation was around 12%, with clear field differences. A 2020 study examined self-citation from 1975 to 2017 and found that it had declined, from roughly 20% in 1975 to under 15% in 2017. High-impact journals do cite their own content more than low-impact journals, but the citations are concentrated in recent work, which is relatively benign.
High self-citation can also reflect editorial manipulation. JCR has removed journals that raised their impact factors by increasing self-citation, but the trick remains common. One familiar example is rejection on the grounds that the manuscript does not cite papers from the journal, and therefore falls outside its research scope. If submitting to a journal effectively requires citing that journal, the policy objectively raises its citation rate. This is a gray zone where motive is hard to prove.
What about open access? In recent years, major publishers have found it useful for launching new journals and building journal portfolios to compete for good submissions, often with the claim that open access raises citations. A 2008 randomized controlled trial in physiology found that open access did not meaningfully increase citations in the year after publication. The paper had open comments, and the authors later reported that follow-up years did not change the conclusion. A 2011 study found similar results in pharmacology, interdisciplinary research, anthropology, and other fields. Open access should not be treated as a citation booster; it is more clearly a publishing-margin model. Its strongest justification is the public's right to know: research paid for by the public should be visible to the public. Paid open access and institutional subscriptions are different charging models, with arbitrage opportunities between them. More promising are fully free and open online publishing models such as the Journal of Machine Learning Research, or self-archiving through preprint servers, though volunteer-run operations are unlikely to be especially efficient. Open access and rising impact factors are separate issues; one should not be blamed for the other.
What about the growth of impact factors themselves? In the 1975 JCR, the highest impact factor belonged to the Journal of Experimental Medicine at 11.874. In 2021, CA-A Cancer Journal for Clinicians reached 286.13. In 1997, only 49 journals had an impact factor above 10; by 2007 there were 105; today there are 660. A look at 11,198 journals over the past five years shows the mean impact factor rising from 2.359 in 2017 to 3.814 in 2021, and the median rising from 1.633 to 2.551. The rise in citations is not in doubt. The causes are multiple.
Global research activity has grown over the past several decades, and the number of researchers in China has risen sharply. More people produce more results, and more results mean more publications. Papers typically cite dozens of references, so more publications generate more citations. Because citations are concentrated in a small number of papers while most papers are barely read, the growth necessarily pushes up the impact factors of leading journals. Modern citation tools and the spread of DOI have also made adding references easy: a few clicks, a quick format change. Studies show the number of references per paper rising in a straight line.
It is hard to prove that software is responsible. If asked why a paper cites forty or fifty works, a researcher is more likely to say they read widely than that the reference manager made it easy. Growth in research activity and citation counts can be viewed as positive developments, but impact factors can also be managed.
Impact factor engineering is a term most journal editors have heard. In recent years, many journals have shifted from academic part-time editing to professional editors, and with that come tactics such as publishing large numbers of items that do not count in the denominator. Some practices are less presentable. A study found that 20% of surveyed scholars believed they had experienced some form of coercive citation, including reviewers demanding citations to related work and editors demanding citations to the journal. The problem is more common at high-impact journals and operates as a tacit exchange. Some requested citations are relevant; others are simply无效 citation. In 2017, the editor of Land Degradation & Development requested 622 additional citations across 82 manuscripts in other journals he oversaw, pushing the impact factor from 3.089 to 8.145. Such crude cases are rare, but in low-volume journals where editors hold strong influence, small maneuvers can inflate a number quickly.
Citation inflation around the impact factor deserves attention, not because the metric is inherently meaningful, but because it raises questions about research evaluation. Researchers should return to the research itself and formulate meaningful scientific questions rather than chase indicators. Success and failure are temporal; truth and falsehood are passed to later generations.
A five-year check is still worth reporting. The dataset covers 11,198 journals with impact factors from 2017 to 2021. Descriptive statistics for citations and impact factors both rise over the period. Linear regression finds that 4,319 journals show a significant linear increase, while only 50 show a significant linear decrease. The average impact factor of the rising journals is below the all-journal average, which suggests a broad rise rather than a few outliers. Among the top 1,000 journals by 2021 impact factor, 522 rose linearly over the five years. Some journals rise in non-linear ways, a trap for linear tests.
Several questions remain open. Do open access journals receive more citations, as the earlier literature suggests they may not? Is there citation spillover between fields, with one field's practices pressing into another? Which fields are declining? Can a better statistic describe journal or individual academic quality? And how should citations outside research articles—news, software, blogs, Twitter, conferences—be summarized? Those are questions for another analysis.