This is a companion post to Most-Cited Health Law Scholars in WestLaw, 2013-2017. As noted there, health law is a broad and fundamentally interdisciplinary field that spans bioethics, biotechnology, medical malpractice, health care finance and regulation, health policy, and public health. The Westlaw citation search partially accounted for this breadth by including leading health law scholars in schools of public health and medicine. However, two major limitations remain—both especially important in our field—which prompted this additional citation analysis.
Westlaw vs. Web of Science
The Westlaw search fails to account for citations in publications that fall outside its law-focused database, which excludes most medical, social science, and public policy journals. Also, a Westlaw search suffers from the “ghost author” problem that attributes multi-authored publications usually to only the first author when there are more than two authors.
To address these issues, we gathered citation counts from the Web of Science database for each of the 40 most highly-cited authors identified in the Westlaw search. Web of Science is generally considered to be the single most comprehensive database of academic publications worldwide. It covers more than 20,000 journals in the sciences, social sciences, and arts and humanities, and it has the added advantage of including more than 90,000 books. Thus, it is likely to be a more accurate and meaningful source than Westlaw for measuring scholarly impact beyond the legal academy. Also, citation counts in Web of Science include all of a publication’s authors, thus addressing the “ghost author” issue that arises in Westlaw searches, as a result of citation conventions in legal periodicals. In addition, Web of Science counts each publication cited, rather than truncating (as the Sisk/Leiter method does) an author’s count to only one publication per article.
All three of the differences mentioned so far—namely, Web of Science’s larger database, its complete author search, and its capacity to count different citations within a single article—mean that Web of Science searches are likely to produce substantially larger citation counts for some authors than searches of Westlaw will.
In another respect, however, Web of Science is more limited than Westlaw. Whereas Westlaw’s Law Journals database has about 1000 journals, Web of Science covers only about 300 legal journals, many of which are international. Also missing are many of the U.S. specialty health law journals. In large part, these limitations are due to that fact that Web of Science emphasizes (but does not strictly require) peer review in selecting which journals to include.
In addition, whereas Sisk’s Westlaw citation searching method is “open,” in the sense that it captures citations to publications in any outlet (including, for example, books or blog posts), search results in the Web of Science are “closed,” meaning that it captures citations only to journals and books in its collection.
A detailed, step-by-step description of the methodology [PDF] we used to search the Web of Science database is described here.
Briefly, using the platform’s Advanced Search interface for its Core Collection database, we entered the last name and first initial for each of the first 40 authors identified by the Westlaw search (using alternative names or initials whenever we identified evidence that these varied over an author’s publishing career). We then manually reviewed the resulting set of possible publications to determine which truly belonged to the intended author, checking the full text of any publications about which there was uncertainty. Once an author’s list of publications was finalized, aggregate citation counts for 2013-2017 were straightforward to calculate because the Web of Science platform automatically provides a count of citations by article and year. We rounded these counts to the nearest 5 (following Leiter).
The results are shown in the table below.
Twenty Most-Cited Health Law Faculty in Web of Science 2013-2017
|3||Mello, Michelle M.||Stanford||2,620||7|
|4||Gostin, Lawrence O. (Larry)||Georgetown||2,550||1|
|5||Greely, Henry T. (Hank)||Stanford||1,595||15|
|6||Rosenbaum, Sara||George Washington||1,410||10|
|7||Hall, Mark A.||Wake Forest||1,145||2|
|8||Annas, George J.||Boston University||1,075||13|
|9||Burris, Scott C.||Temple||915||20|
|11||Sage, William M.||Texas||525||17|
|12||Cohen, I. Glenn||Harvard||425||4|
|13||Outterson, Kevin||Boston University||345||>20|
|14||Hodge, James G., Jr.||Arizona State||300||>20|
|19||Robertson, Christopher||University of Arizona||155||>20|
|20||Francis, Leslie P.||Utah||150||>20|
As the table shows, Web of Science searches generated substantial differences in health law citation counts compared with Westlaw-based rankings. While this reflects the broad scope of our particular field, it is possible that significant differences would also be observed in other fields of legal scholarship, when looking beyond primarily legal publications.
 While Web of Science search corrects the problem in the Westlaw search of failing to count authors other than the first author in publications with 3 or more authors, the fix exposes another issue: equal weighting of authors. Consider, for example, a 6-author Science article that is heavily cited. A citation to the article counts as 1 for the third and fourth authors of that article, just as it does for the article’s first and last (senior) authors, and just as it would for a solo author of another publication. When author order is determined alphabetically, as it often is in multi-author publications in social science journals, this may be appropriate. But when authorship order indicates a priority of intellectual contribution to the work, as it often does in multi-author publications in medical and health policy journals, an equal count for all authors of the article may be inappropriate. Attributing credit in multi-author publications arising out of collaborative research has long been a vexing issue in medical and public health scholarship; it poses challenges in many areas, ranging from evaluations of publication quality to hiring and promotion decisions. There is no consensus on how to resolve it. The problem’s complexity is compounded by heterogeneity in practices across research teams and journals, even within the health and medical sciences. For purposes of our current exercise, it is a timely reminder that citation counts, measured by whatever method, are an imperfect measure of a scholar’s impact in the publishing world, and, of course, even if that impact could be measured precisely, it is only a piece of the larger question of what we ought to care about.
 However, Web of Science does not count multiple citations to the same publication in a single article; instead, it gives each publication only a single count, no matter how many times an article cites it.
 Included are American Journal of Law & Medicine; Journal of Health Politics, Policy and Law; Journal of Law and The Biosciences; Journal of Law, Medicine & Ethics, and Journal of Legal Medicine.