How do hydrology and water resources journals compare to each other? This question, as noted by McDonnell [1997], is frequently asked by reappointment, tenure, and promotion committees as well as by researchers and graduate students who wonder where to submit their papers. Although the quality of a journal clearly rests with the articles that it publishes, not vice versa, the quality of one's work is often judged in terms of the quality of the journal in which it is published. Therefore, ignoring the journal rankings may be costly regardless of whether or not we deplore the obsession of our society with rankings. What constitutes then a prudent journal choice? Should it be based on the elusive reputation of a journal, its selectivity, its most recent impact factor (defined in the next section), or other statistics? Although Dagan [1989] and McDonnell [1997] have provided rankings of selected journals by the impact factor, they have not answered these questions and neither will this author. Below, their data and discussion are expanded
Citation StatisticsJournal Citation Reports, published annually by the Institute for Scientific Information in Philadelphia, Pennsylvania, provides a wealth of statistical information about all sufficiently large or established journals. Among the citation statistics, three are of particular relevance: Total Citations, Cited Half-Life, and Impact Factor.
Total Citations (TC) is the number of all citations of source items (papers, comments, etc.) published in the journal during a given year. A large TC may indicate a journal's dominance in the field. It is also characteristic of multidisciplinary journals (such as Nature) and journals catering to broad fields (such as Journal of Geophysical Research). A small TC may indicate a journal's weakness or less frequent publication. However, it is also typical for narrow-specialty journals even of high quality.
Cited Half-Life (CHL) is the number of journal publication years going back from the current year which account for 50% of the total citations received by the cited journal in the current year. It indicates the age of its average cited articles. A large CHL may imply longevity of the published information. However, a relatively small CHL may reflect a journal's emphasis on cutting edge research and its timeliness.
Impact Factor (IF) is the ratio between the number of all current citations of source items published in a journal during the previous 2 years to the total number of articles that the journal published during that time. It can also be viewed as the frequency with which the "average article" in the journal has been cited in a particular year. The IF reflects the influence on its field of an average article published in the journal. It is also believed to be a fair quality measure that "tends to discount the advantage that large, frequently issued, older journals have over smaller, less frequently issued, newer journals" [McDonnell, 1997]. Of the three statistics, the impact factor is generally considered to be best correlated with a journal's quality, an elusive concept not defined here.
Although the citation statistics are often interpreted as measures of journal quality, the Institute for Scientific Information [1995] warns that "The quantitative data it reports are intended to complement, not replace, traditional quantitative and subjective inputs, such as peer surveys and specialist opinions." Dagan [1989] further cautions "...citation habits differ among fields, and comparison of journals belonging to different areas is not warranted." This may also be true even for the journals covering the same field, yet catering to different audiences.
Of course, the citation statistics cannot be used to judge the quality of a single article! They only reflect the nature of an average article. Thus one can expect to see some poor-quality contributions published in the top journals as well as gems published in low-ranked journals.
Rankings Based on Citation Statistics
Table 1 lists citation statistics, the starting year, the publisher, and information about page charges for journals whose focus is mainly or significantly oriented toward hydrology and water resources. However, two petroleum engineering journals are also included because characterizations of petroleum reservoirs and aquifers are technically analogous. Although the journals displayed in Table 1 are ordered by the average impact factor (1991-1996), this is not meant to imply that this is the ultimate journal ranking. The last seven journals in Table 1 (included because they publish some hydrology-related research) have very different audiences and likely very different citation habits; comparing their statistics to those of the remaining journals is thus unwarranted. Similarly, comparing citation statistics of Ground Water and Ground Water Monitoring & Remediation to those of the remaining journals may be questionable because the two journals have significantly different audiences (with a much higher fraction of authors from industry and a correspondingly smaller fraction of authors from academia than the remaining journals) and, consequently, possibly significantly different citation habits. Comparison of Stochastic Hydrology and Hydraulics to the other journals in terms of the citation statistics may also be questionable due to its highly specialized nature. The same reason makes the interpretation of statistics for Ground Water Monitoring & Remediation even more difficult.
| Table 1. Statistics of hydrology and water resource journals. IF = impact factor, TC = total citations, CHL = citation half life, and Avg = average. |
| Click here to view enlarged table (~99 Kbytes) | |
| * Double horizontal lines set apart journals of significantly different audiences to make comparison of the statistics unwarranted. 1 Blank means no page changes 2 Narrow-specialty journal; comparison of its impact factor to the impact factors of other journals is questionable 3 The average calculated from less than 5 years 4n/a = not available 5 Journal of sufficiently different audience to make questionable the comparison of its impact factor to those of other journals. |
With these qualifications, note that among the hydrology and water resources journals Water Resources Research and Journal of Contaminant Hydrology are the overall citation-statistics leaders. Water Resources Research has the highest IF averaged over any 6-year, 5-year, 4-year, 3-year, or 2-year period during the years 1991-1996, as well as the highest IF during 4 out of the 6 years. In addition, it has the highest number of total citations in 1996. In fact, its TC is more than 3 times higher than that of the next journal and 1.25 times higher than the combined total citations of all the other (11) hydrology journals. Journal of Contaminant Hydrology, on the other hand, claims the highest IF in 1996 and 1994. The impact factors for the two journals averaged over 3 years (1994-1996) are close to each other (1.721 and 1.707, respectively). However, Nordic Hydrology claims the highest CHL in 1996 and is followed closely by Hydrologic Sciences Journal, the oldest among the hydrology journals, as well as Water Resources Bulletin and Water Resources Research. Further ranking is not at all clear. The third overall spot in citation-statistics ranking has no fewer than four contenders: Ground Water with the third highest IF and the third highest TC (but comparison of these statistics to those of other journals may be questionable, as explained earlier), Transport in Porous Media with the next largest 6-year average IF as well as the next largest IF in 1996, Journal of Hydrology with the second largest TC and the IF larger than that for Transport in Porous Media in 3 out of 6 years, and Hydrological Processes with the higher 4-year average IF in 1992-1995 and 1993-1996 than the last two journals and the second highest IF in 1993. One should also note that the IFs of Advances in Water Resources, Water Resources Bulletin, and Stochastic Hydrology and Hydraulics are rising in recent years, approaching those of the above mentioned journals. In fact, the IF for Advances in Water Resources rose in 1996 above that of Journal of Hydrology and Hydrological Processes. Some of these journals along with Ground Water Monitoring & Remediation (IF-ranked below Ground Water and thus not considered for the third spot) are also contenders to the fourth spot. What is then the true ranking of journals in hydrology and water resources? This question is left for the reader to resolve.
The impact factors versus time are plotted in Figure 1 for 12 hydrology journals. It is clear from the presented data that the IF and the IF ranking may exhibit significant fluctuations from year to year. These fluctuations may be partially explained by publication of special journal issues devoted to narrow topics, which tends to lower the journal IF in an analogous manner to that of narrow-specialty journals in spite of the high quality of the invited papers. The fluctuations may also be partially explained by publication of a varying number of review articles, which are usually cited much more often than the focused research articles and thus tend to increase the journal IF. This is especially evident from the high IF of the review journals such as Reviews of Geophysics and Annual Review of Fluid Mechanics, presented later.
The reader should note that in most cases the smaller the total citations, or roughly the smaller the journal, the larger are the IF fluctuations. Therefore, McDonnell's [1997] belief in the particular fairness of the IF ranking for evaluation of relatively new or small-circulation journals may be questionable. The reader should also note that the impact factor rankings at any given year are likely to change upon elimination of self-citations, which many believe do not fully contribute to the measure of the journal's impact on its field. Garfield [1994] gives a relevant example and points out that self-citations often reach as much as 13% of all citations a journal receives.
Ranking Based on Journal's Selectivity
Another measure of journal quality is the manuscript selectivity reflected by the journal's acceptance rate, which is the ratio of accepted papers to the total number of papers considered for publication in a given time period, or by the related rejection rate. The lower the acceptance rate, the more selective is the journal and, presumably, the higher is its quality. Some believe that rather than the IF or any other citation-statistics measure, the selectivity of a journal is best correlated with its quality. However, there are at least four problems with this quality measure.
First, two journals that publish papers of the same quality may have appreciably different acceptance rates. This could be the case if the two journals adopted the same standards for manuscript acceptance, yet different ones for rejection. For example, one journal may reject a paper requiring major revisions and simultaneously encourage resubmission of its revised version (for instance, Water Resources Research), whereas the other journal may allow major revisions while still imposing the same quality control through the second partial or full review (for instance, Journal of Contaminant Hydrology). The first journal would count the ultimately accepted paper twice, once as a rejection and once as an acceptance, whereas the other journal would count it only once. This will obviously result in different acceptance rates in spite of the same acceptance standards.
Second, although a manuscript is usually rejected for good reasons, often it does not have a fatal flaw. Such a manuscript benefits from the scrutiny of the review process and in the revised form will frequently meet the quality standards of the journal that originally rejected it. Yet, often the revised and improved manuscript is submitted to another journal, which accepts it "as is" or subject to minor revision. In the process, the original journal "earns" selectivity points, whereas the second journal "loses" them at the "expense" of getting a high-quality contribution.
Third, publication of special issues with invited papers written by some of the best authorities in the field and thus overwhelmingly accepted for publication tends to dilute the journal's selectivity in spite of the high quality of the papers.
And fourth, the number of rejected papers may be counted differently by different journals. For example, in some journals, a rejected paper is often ultimately accepted in a revised form. In others, a paper "accepted with major revisions" may never be published because the required changes are just too onerous. Such a paper may be counted as "accepted" by some journals or as "rejected" by others. In addition, the editorial offices of some journals return to the authors a number of submitted manuscripts, due to their poor quality or local interest, before they even reach the editors. This is the case with Journal of Hydrology and Journal of Contaminant Hydrology, which do not count these returns in their rejection rates. According to Frans Koning [personal communication, 1997] from the editorial office of these journals, counting or discounting these manuscripts may make a difference of about 10% in the rejection rate.
Obtaining selectivity data for all the considered journals proved to be difficult; only some editorial offices provided their acceptance rates. Figure 2 displays the annual acceptance rates for Water Resources Research, Journal of Contaminant Hydrology, Journal of Hydrology, Environmental Science & Technology, Soil Science Society of America Journal, and Geophysical Research Letters, which are among the top journals in their own fields. Although the presented selectivity data are limited, they allow for two observations. First, even for the large-circulation journals, selectivity is subject to fluctuations (one would expect larger fluctuations for small-circulation journals). Second, the selectivity and the impact factor of a journal clearly do not correlate perfectly. For example, Journal of Hydrology has the long-term acceptance rate of "about 50%," which is lower than that of Journal of Contaminant Hydrology by about 30% and yet the latter journal has a much larger IF than the former (see Figure 1 and Table 1). This is especially intriguing because the two journals are not only reviewed by similar pools of reviewers but are also handled by the same editorial office. Frans Koning [personal communication, 1997] believes that papers submitted to the latter journal are on the average of a higher quality than those submitted to the former journal, "so you cannot compare the rejection rates."
Reputed Authors' Publication Habits
Although elusive, scholarly reputation is probably the most important measure of journal quality. It can be established by polling a large enough sample of authors (AGU might consider organizing such a poll). However, a frequency with which a representative group of authors of high scholarly reputation publishes in a given journal can be estimated without polling and can arguably be treated as an indicator of a journal's reputation. Even though this indicator is intrinsically biased - it reflects where the reputed authors publish, but not necessarily where they would prefer to publish - it may be of interest.
Identification of a representative group of authors for quantitative assessment of publishing habits is admittedly a subjective exercise. In fact, it is doubtful that a consensus could ever be reached on the composition of such a group in terms of representation of different hydrological subfields or even on what constitutes the modern field of hydrology. However, to examine the journal choices of respected hydrology researchers for publication of their own work, a group of 57 established researchers has been selected (mostly from universities, national laboratories, and the U.S. Geological Survey). The researchers are active mainly in subsurface hydrology and, particularly, in the areas of hydraulic and chemical subsurface characterization, water and contaminant transport in porous or fractured formations, multiphase flow through porous media, and subsurface remediation. This group is not meant to be inclusive of all high-quality hydrology research, not even all subsurface hydrology research, but was chosen as a representative sample. Unfortunately, the selection is necessarily biased by this author's journal reading habits. The results should therefore be interpreted accordingly.
The journal publications of the representative group between the years 1989 and 1995 were searched through MELVYL, the University of California Online Catalog (based on Current Contents). The search yielded 1313 publications, with multiple counting of the multi-author papers. The multiple counting was not eliminated; it was welcomed instead as a reflection of the "right to vote" for a journal by all the coauthors who belong to the representative group. The results are presented in Table 2.
| Table 2. Publication habits of a sample of 57 researchers active in areas related to subsurface hydrology. |
 |
 |
Water Resources Research again dominates overwhelmingly, whereas Journal of Hydrology holds a solid second place, with Advances in Water Resources and Ground Water arguably sharing third place. They are closely followed by the equally popular Journal of Contaminant Hydrology and Transport in Porous Media. Publication habits of the representative group clearly differ from the IF ranking. For example, they give much more prominence to Advances in Water Resources and Stochastic Hydrology and Hydraulics than the IF ranking. However, both measures confirm the significant impact of the journals with relatively high IF on the subsurface-hydrology research community.
The reader should note that the publication habits of a group of scientists whose research has a different focus than that of the group considered here would likely be different, and so would be the publication habits of a group of practitioners.
Comparison of Rankings
The considered rankings for hydrology journals are compared to each other in Table 3. As explained earlier, the rankings by the impact factor for Ground Water and Ground Water Monitoring & Remediation as well as Stochastic Hydrology and Hydraulics may be misleading because the first two journals have significantly different audiences than the other ones (with likely different citation habits), whereas the last journal is of a highly specialized nature and, consequently, has a relatively low impact factor.
It turns out that the considered rankings are generally inconsistent with each other and with publication habits of reputed authors. Establishing an overall ranking for all the journals is thus difficult, if not impossible. However, it is clear that Water Resources Research is the overall leading hydrology journal. In rankings according to the impact factor, it is followed closely by Journal of Contaminant Hydrology.
| Table 3. Comparison of the rankings for subsurface journals. IF stands for impact factor, TC for total citations, CHL for citation half life, Avg for averaged over years 1991-1996, WRR for Water Resources Research, JCH for Journal Containment Hydrology, GW for Ground Water, GWM&R for Ground Water Monitoring & Remediation, TiPM for Transport in Porous Media, JH for Journal of Hydrology, HP for Hydrological Sciences Journal, SH&H for Stochastic Hydrology and Hydraulics, and NH for Nordic Hydrology |
 |
| 1 Journal of sufficiently different audience to make questionable the comparison of its impact factor to the impact factors of other journals. 2Narrow-specialty journal; comparison of its impact factor to the impact factors of other journals is questionable. |
Other Factors
A journal choice may also be affected by factors other than ranking or reputation; these factors include the exact nature of the manuscript and the journal's niche, time taken to publish, and page charges.
The best journal to publish a manuscript in a broad field such as hydrology and water resources may depend upon the exact nature of hydrologic research that the manuscript is reporting. Although all discussed journals undeniably publish a wide spectrum of papers, they nevertheless have developed their own niches. For example, Ground Water, Transport in Porous Media, and Advances in Water Resources seem (or used) to have an applied, theoretical, and computational flavor, respectively. This is also true of the two hydrologic journals with the highest IF--some important areas of hydrology are not well represented in them. For example, Journal of Contaminant Hydrology does not publish contributions solely related to water resources, aquifer characterization, and flow through porous media, whereas Water Resources Research publishes hydrologic contributions related to geochemistry relatively infrequently. Based on the title-word search of the earlier mentioned database MELVYL, hydrologic geochemistry research seems to be published mostly in Journal of Hydrology, Geochemica et Cosmochimica Acta, and Environmental Science-Technology, whereas hydrologic remediation and bioremediation research seems to be published mostly in Ground Water and Ground Water Monitoring & Remediation. In addition, some journals favor or discourage certain kinds of papers. For example, requiring a significant contribution to hydrology and water resources, Water Resources Research has been discouraging papers focusing solely on the development of novel numerical or analytical methodologies [Hornberger, 1994], whereas other journals may welcome such papers. Thus the nature of the manuscript may indeed limit the journal choices.
The average time to publish an article differs significantly from journal to journal. The average time for peer review varies and the time from acceptance to appearing in print does vary significantly between journals. In fact, at Water Resources Research it takes about 4 months to publish an accepted paper, whereas at other journals it takes much longer--for example, based on the last regular issue available in August 1997, 6 months in Transport in Porous Media, 8 months in Advances in Water Resources, 10 months in Journal of Contaminant Hydrology as well as in Ground Water, 11 months in Hydrological Processes, and 13 months in Journal of Hydrology.
The average time to publish an accepted paper in a given journal may fluctuate. For example, as opposed to the mentioned 13 months, it is only 8 months on the average for Journal of Hydrology [Frans Koning, personal communication, 1997]. Although it may seem that Water Resources Research has by far the speediest publication cycle, the differences between the journals in the average time to publish may not be as large when corrected for the second full review that a paper requiring significant revisions is often forced to undergo at Water Resources Research. In fact, many authors would rather have a paper in press for a longer period awaiting publication rather than being in limbo with continued reviews.
In a climate of tightening research budgets and a number of quality journals publishing at no cost to authors, the journal choice may come down to page charges, especially when they begin to exceed the cost of a decent personal computer. Indeed, the publication of a 15-page article carries mandatory page charges of $2,025 in Ground Water, $1,600 in Ground Water Monitoring & Remediation, $1,755 in Water Resources Research, and $1,155 in Water Resources Bulletin. Corresponding costs for publication of a 10-page article are $1,000, $600, $430, and $730, respectively. The longer the articles, the more tempting is its submission to a quality journal that has no page charges, such as journals published by Elsevier, Kluwer, Wiley, and others (see Table 1).
Tables 2 and 4 provide statistics for journals in fields broadly related to hydrology and water resources, such as environmental sciences and engineering, soil sciences, fluid mechanics, geophysics, geotechnical engineering, and applied mathematics. The IF ranking and the publication-habits ranking (based on the same group of researchers) agree in identifying the top journal not only for the area of hydrology and water resources, but also for environmental science and engineering as well as geoscience. This is also the case for soil science if the two top IF journals in this field are considered to be equivalent. However, the two rankings disagree again further down the scale. The full interpretation of the information contained in the two tables is left to the reader.
ConclusionsA number of general conclusions can be reached. First, the choice of the right journal for a manuscript can be affected by many factors including the exact nature of the manuscript; the journal's scope, reputation, and rankings; the length of the time lag between acceptance and publication; and page charges. Second, none of the considered quality measures could be said to correlate perfectly with the elusive concept of journal quality and thus none should be used as a sole criterion for journal selection. Third, rankings of hydrology and water resources journals based on citations statistics and journal selectivity fluctuate significantly from year to year and are generally inconsistent with each other. These rankings are also generally inconsistent with publication habits of reputed authors. Fourth, it is clear that although some of the existing journals cover the field more broadly than do others, none could be considered a leading journal in all subfields of hydrology and water resources. However, an overwhelming majority of the presented rankings and data confirm that among a host of high-quality journals, Water Resources Research is the leading journal for many of the subfields. In rankings according to the impact factor, it is followed very closely by Journal of Contaminant Hydrology.
| Table 4. Statistics for journals broadly related to Hydrology and Water Resources. IF = impact factor, TC = total citations, and CHL = citation half life. |
 |
 |
 |
I am grateful to the colleagues who provided numerous constructive and critical comments, which influenced the evolution of my views on the subject. Feeling uneasy about parts or even the entire article, many of these colleagues requested anonymity. Since this article presents my interpretations of the existing data and my own opinions, some of which may be controversial, I do not acknowledge anyone by name.
References