In June 2019, Elsevier announced a co-development agreement with NETE, a digital solutions company focusing primarily on providing research assessment to federal government agencies. Two weeks later, Elsevier announced the launch of an International Center for the Study of Research (ICSR), a venture designed to “cultivate the thoughtful use of metrics and indicators in research evaluation and to promote evaluation best practices.” ICSR is tasked with “developing, characterizing and validating new and existing research metrics, indicators and research assessment practices; it also supports independent, external studies on topics within this scope.”
These two announcements were largely ignored by the media, but we believe that they raise significant issues:
-
Conflict of interest. There is an inherent conflict of interest between assessing and evaluating research, on the one hand, and disseminating it, on the other. We know that all kinds of biases and errors slip into algorithms, even unintentionally. There may be little or no way for federal agencies to identify whether any biases creep into the NETE evaluations and favor publishing research in Elsevier journals. Similarly, there may be no way for academic institutions or funding bodies to assess whether there are any biases embedded into the methodologies and standards proposed by ICSR. In aggregate, federal agencies continue to be the largest source of funding for STM research in the US.
A deeper relationship between Elsevier and these agencies heightens the risk of excessive concentration in the provision of data analytics to academic institutions as universities are likely to gravitate toward accessing the evaluation tools used by their largest source of funding. The more successful NETE is with its customers, the stronger the motivation for academic institutions to access its data. This is likely to reduce the opportunities for other players to compete with Elsevier in research assessment analytics.
-
Excessive control. Perhaps the greatest source of concern is the statement contained in the Elsevier/NETE press release highlighting the aspiration to offer “seamless solutions for managing research information.” It is laudable to pursue interoperability across research information systems, but it is risky to put these capabilities into the hands of one company which may find itself at the center (and perhaps in control) of a large percentage of all research dissemination and assessment data flows.
In general, the academic community should approach any companies driving the definition of research evaluation methodologies and/or performing actual assessments when they have clear conflicts of interest with a healthy dose of skepticism. In the case of ICSR, these concerns are heightened by the announcement in March 2020 that it will operate a computational platform, ICSR Lab, which will host researchers wanting to analyze large data sets (including SCOPUS and PlumX, but also their own).
Users of ICSR Lab will maintain ownership of their own data sets, but they must grant ICSR and Elsevier the right to use the data fed by researchers “for the purpose of developing and maintaining the ICSR Lab and related services,” a formulation that is vague enough to allow ICSR (and possibly Elsevier) a wide and potentially unlimited range of uses. At a minimum, Elsevier and ICSR will have access to a significant amount of analytical work on science assessment and be in a position to affect the outcome of research projects on science assessment.
