UNIVERSITY

Webometrics

Webometrics

Webometrics 

The Ranking Web or Webometrics is the largest academic ranking of Higher Education Institutions. Since 2004 and every six months an independent, objective, free, open scientific exercise is performed by the Cybermetrics Lab (Spanish National Research Council, CSIC) for the providing reliable, multidimensional, updated and useful information about the performance of universities from all over the world based on their web presence and impact.

History

The Cybermetrics Lab has been developing quantitative studies on the academic web since the mid-nineties. A first indicator was presented during the EASST/4S conference in Bielefeld (1996) and the collection of web data from European universities started in 1999 supported by the EU funded project EICSTES. These efforts are a follow-up of our scientometric research started in 1994 that has been presented in the conferences of the International Society for Scientometrics and Informetrics (ISSI, 1995-2011) and the International Conferences on Science and Technology Indicators (STI-ENID, 1996-2012) and published in high impact journals (Journal of Informetrics, Journal of the American Society for Information Science and Technology, Scientometrics, Journal of Information Science, Information Processing & Management, Research Evaluation and others). In 1997 we started the edition of an all-electronic open access peer-reviewed journal, Cybermetrics, devoted to the publication of webometrics-related papers.

In 2003 after the publication of the ShanghaiJiatongUniversitybreakthrough ranking, the Academic Ranking of World Universities (ARWU), we decided to adopt the main innovations proposed by Liu and his team. The ranking will be built from publicly available web data, combining the variables into a composite indicator, and with a true global coverage. The first edition was published in 2004, it appears twice per year since 2006 and after 2008 the portal also includes webometrics rankings for research centers, hospitals, repositories and business schools.

Objectives and motivation

The original aim of the Ranking is to promote academic web presence, supporting the Open Access initiatives for increasing significantly the transfer of scientific and cultural knowledge generated by the universities to the whole Society. In order to achieve this objective, the publication of rankings is one of the most powerful and successful tools for starting and consolidating the processes of change in the academia, increasing the scholars’ commitment and setting up badly needed long term strategies

The objective is not to evaluate websites, their design or usability or the popularity of their contents according to the number of visits or visitors. Web indicators are considered as proxies in the correct, comprehensive, deep evaluation of the university global performance, taking into account its activities and outputs and their relevance and impact.

At the end a reliable rank is only possible if the web presence is a trustworthy mirror of the university. In the second decade of the 21st century the Web is key for the future of all the university missions, as it is already the most important scholarly communication tool, the future channel for the off-campus distance learning, the open forum for the community engagement and the universal showcase for attracting talent, funding and resources.

Philosophy and justification

Webometrics only publish a unique Ranking of Universities in every edition. The combination of indicators is the result of a careful investigation and it is not open to individual choosing by users without enough knowledge or expertise in this field. Other publishers provide series of very different rankings using exactly the same data in different fashions that is completely useless and very confusing.

Webometrics is a ranking of all the universities of the world, not only a few hundred institutions from the developed world. Of course, “World-class” universities usually are not small or very specialized institutions.

Webometrics is continuously researching for improving the ranking, changing or evolving the indicators and the weighting model to provide a better classification. It is a shame that a few rankings maintain stability between editions without correcting errors o tuning up indicators.

Rankings backed by a for-profit company exploiting rank-related business or with strong political links reflected in individual ranks should be checked with care.

Research only (bibliometrics) based rankings are biased against technologies, computer science, social sciences and humanities, disciplines that usually amounts for more than half of the scholars and students in a standard comprehensive university. Webometrics also measure, in an indirect way, other missions like teaching or the so-called third mission, considering not only the scientific impact of the university activities, but also the economic relevance of the technology transfer to industry, the community engagement (social, cultural, environmental roles) and even the political influence.

Webometrics uses link analysis for quality evaluation as it is a far more powerful tool than citation analysis or global surveys. In the first case, bibliometrics only counts formal recognition between peers, while links not only includes bibliographic citations but also third parties involvement with university activities. Surveys are not a suitable tool for World Rankings as there is not even a single individual with a deep (several semesters per institution), multi-institutional (several dozen), multidisciplinary (hard sciences, biomedicine, social sciences, technologies) experience in a representative sample (different continents) of universities worldwide.

Research output is also key topic for Webometrics, but including not only formal (e-journals, repositories) publications but also informal scholarly communication. Web publication is cheaper, maintaining the high standards of quality of peer review processes. It could also reach much larger potential audiences, offering access to scientific knowledge to researchers and institutions located in developing countries and also to third parties (economic, industrial, political or cultural stakeholders) in their local community.

We intend to motivate both institutions and scholars to have a web presence that reflect accurately their activities. If the web performance of an institution is below the expected position according to their academic excellence, university authorities should reconsider their web policy, promoting substantial increases of the volume and quality of their electronic publications.

Candidate students should use additional criteria if they are trying to choose university. Webometrics ranking correlates well with quality of education provided and academic prestige, but other non-academic variables need to be taken into account.

Composite indicators and Web Impact Factor

Probably one of the major contributions of the Shanghai Ranking was to introduce a composite indicator, combining with a weighting system a series of indicators. Traditional bibliometric indexes are built on ratios like the Garfield’s Journal Impact Factor that based on variables following power law distributions is useless for describing large and complex scenarios. The Ingwersen proposal in 1997 for a similarly designed Web Impact Factor (WIF) using a links/webpages (L/W) ratio is equally doomed by the mathematical artifacts that generates.

Following the Shanghai model we developed an indicator transforming the ratio L/W into the following formula aL+bW, where L & W should be normalized in advance and a & b are weights adding 100%. We strongly discouraged the use of WIF due to its severe shortcomings. The composite indicator can be designed with different sets of variables and weightings according to the developer’s needs and models.

Design and Weighting of Indicators

Webometrics uses an “a-priori” scientific model for building the composite indicator. Other rankings choose arbitrary weights for strongly dependent variables and even combine raw values with ratios. None of them follow a logical ratio between activity related and impact related variables, i.e. each group representing 50% of the total weighting. Referring to the individual variables, some of them have values larger than zero for only a few universities and others segregate universities according to differences so small that they are even lower than their error rates. Prior to combination the values should be normalized, but the practice of using percentages is mostly incorrect due to the power law distribution of the data. Webometrics log-normalize the variables before combining according to a ratio 1:1 between activity/presence and visibility/impact groups of indicators. The current composite indicator is now built as follows:

Visibility (50%)

IMPACT. The quality of the contents is evaluated through a "virtual referendum", counting all the external inlinks that the University webdomain receives from third parties. Those links are recognizing the institutional prestige, the academic performance, the value of the information, and the usefulness of the services as introduced in the webpages according to the criteria of millions of web editors from all over the world. The link visibility data is collected from the two most important providers of this information: Majestic SEO and ahrefs. Both use their own crawlers, generating different databases that should be used jointly for filling gaps or correcting mistakes. The indicator is the product of square root of the number of backlinks and the number of domains originating those backlinks, so it is not only important the link popularity but even more the link diversity. The maximum of the normalized results is the impact indicator.

Activity (50%)

PRESENCE (1/3). The total number of webpages hosted in the main webdomain (including all the subdomains and directories) of the university as indexed by the largest commercial search engine (Google). It counts every webpage, including all the formats recognized individually by Google, both static and dynamic pages and other rich files. It is not possible to have a strong presence without the contribution of everybody in the organization as the top contenders are already able to publish millions of webpages. Having additional domains or alternative central ones for foreign languages or marketing purposes penalizes in this indicator and it is also very confusing for external users.

OPENNESS (1/3). The global effort to set up institutional research repositories is explicitly recognized in this indicator that takes into account the number of rich files (pdf, doc, docx, ppt) published in dedicated websites according to the academic search engine Google Scholar. Both the total files Both the total records and those with correctly formed file names are considered (for example, the Adobe Acrobat files should end with the suffix .pdf). The objective is to consider recent publications that now are those published between 2008 and 2012 (new period).

EXCELLENCE (1/3). The academic papers published in high impact international journals are playing a very important role in the ranking of Universities. Using simply the total number of papers can be misleading, so we are restricting the indicator to only those excellent publications, i.e. the university scientific output being part of the 10% most cited papers in their respective scientific fields. Although this is a measure of high quality output of research institutions, the data provider Scimago group supplied non-zero values for more than 5200 universities (period 2003-2010). In future editions it is intended to match the counting periods between Scholar and Scimago sources.