2.1. Citation Rate
Several measures of publication quality are studied in the journal ranking literature, each with its own strengths and shortcomings. Mainly, the measures fall into three categories: those based on citations, those based on acceptance rates, and those based on rankings of journals by professionals with some relevant expertise. Each is a proxy measure for journal quality; each is used to develop ordinal rankings or groups of “top” journals. Journal rank may in turn be used as a proxy for article quality, or author quality, or institution quality. How well does each type of measure reflect value? Are they consistent?
Citation is an indication that one scholar’s work is valuable to someone else. More citations should indicate higher value. MacRoberts & MacRoberts (1989) review some problems of citation analysis. Different rates of citation are customary in different fields. Multiple authors of an article are not always credited. There may be biased citing particularly when secondary sources are cited rather than initiators. Informal sources and tacit knowledge are usually not cited because they are not in papers and books. There is self-citation. Some citations are negative while others are positive. Not all literature is included in the SSCI which is the main source of citation impact factors. New journals and journals in specialized areas have fewer citations than general purpose and older, better known journals. When analyzing citation patterns, different authors use different length publication year windows for impact factors, making comparisons difficult. The most important papers are often not cited because their results are taken for granted to be part of the norm (Ritzberger, 2008); his example, users of the Nash equilibrium rarely cite Nash (1950). New developments are often in new journals which take a long time to get into citation indexes. The peer review system of journals is biased against authors that are not affiliated with top universities or are employed at non-academic institutions (Blank, 1991). Anthony F. J. Van Raan (2005) discusses some bibliometric problems of using citation index as a proxy for research quality, enumerating various possible sources of error in collection of the data used to compute a citation index. Citation is a complex process, practice differs among fields; he suggests citation is improper for evaluating research even aggregated at the level of large institutions. He advocates for improving bibliometric indicators and using them as support for, rather than replacement of, a peer-based evaluation procedure.
Citation rate can be computed for a journal as well as for an individual paper. Thus, citation rates are sometimes used in studies that compare journal quality and article quality.