Menu Close

Frames on Brazilian Research

This text was originally published in the Jornal da Unicamp. Click here to read the original text in portuguese.


In August of this year, a report by Clarivate and the Coordination for the Improvement of Higher Education Personnel (Capes) was released, detailing the latest on Brazilian scientific production[1]. The document, entitled Panorama das Mudanças na Pesquisa no Brasil (Overview of Changes in Research in Brazil), was sponsored by Capes, the institution responsible for overseeing postgraduate education in the country and for producing the majority of Brazil’s scientific output. As is well known, everything that Capes does, supports, and disseminates has an impact and signals values for research in the country.

Hence, the interest in understanding what the report reveals.

The report is an analysis of publications indexed in the Web of Science (WoS), supplemented by data from other Clarivate databases.

Basically, all the information in the report can be accessed by anyone with access to these databases (and many organizations in Brazil have this access). WoS is a global reference, alongside Scopus (Elsevier) and Dimension (Digital Science), or even open and free databases like OpenAlex and Scielo.

A first observation is that these databases vary in scope, with some being more comprehensive than others. A second observation is that open and free databases already exist and provide material that is little or no different from closed, paid ones. Scientific journal coverage also varies greatly, with some journals being much more widely covered than others, sometimes by a factor of two or three, depending on the reference[2].

Using different sources, which have significant differences, tends to lead to different data and interpretations. The idea that science is global should be true, but it is a point of escape: it depends on the observer. As has been discussed and shown in the specialized literature, some themes, approaches, geographies, institutions, and publishers are “more global” than others.

Therefore, the first caution the reader should take is that the Clarivate/Capes report comes from a data selection that someone considered important. A snapshot that chooses a certain angle, with a certain resolution. The document is far from (and could not be) offering a comprehensive and in-depth portrait of Brazilian research.

This warning should be on the cover page of the report.

The document presents updated information on some well-known indicators within the community. It also brings original information that catches attention and sparks interest for future research, as it will only be possible to understand what these data mean after further investigation.

By the WoS measure, Brazil is maintaining its 13th place globally (in terms of publications), but the relative importance measured by citation counts (normalized by field and period) has stayed below the median compared to the G7 (which is expected), the BRICS (expected for some of them), and Latin America (less expected).

Further on, the report, in Figure 26, analyzes Brazil’s “performance” in ten “key” global citation topics. The figure shows that of the topics selected from WoS, six are classified as weaknesses and two as relative threats. Only two remain as strengths for the country: antibiotic resistance and coronaviruses[3].

The question remains: does this combination of information mean we are becoming more productive (or maintaining production) and less relevant? Does this conclusion make sense? Are the data and analysis used the best way to look at the evolution of research in Brazil?

Regarding open access publications, according to the report, Brazil is relatively stable, with a slight increase in the Gold type, meaning the article is open but a processing fee (APC) was paid to the publisher. It’s unfortunate that the report doesn’t separate fully open access publications that are free of APCs (known as Diamond access). According to the report, among 20 countries selected for having the highest scientific output globally[4], open access percentages range from 36% (India) to 84% (Netherlands), with Brazil at 54%. The median for this group is 60%.

Another question: should we be publishing more in open access? Since several studies show they have greater impact, not just in terms of citation counts, but also, and perhaps more importantly, in the diversity of readers, mentions, citations, and users of knowledge?[5]

On international collaboration, measured by co-authorship with researchers from other countries, Brazil has made progress. From less than 30% ten years ago, we are now at 38%, with 13% of collaborations with the United States, 6% with the United Kingdom, and 4% with Spain and Germany. The median for the same 20 countries is around 45%, ranging from 23% to 75%.

One more question: should we have advanced further in international collaborations? Both in total and in the diversity of countries and collaborator profiles?

These questions become less clear when we reach Table 15, which presents bibliometric indicators comparing Brazilian publications funded by national and foreign agencies.

Basically, the table leads the reader to believe that Brazilian research only has impact when funded by foreign agencies. The differences are so large that one would expect a warning to the reader, such as:

“Attention: these data, presented as they are, without weighting or discussion, do not mean that Brazilian agencies fund research with less impact than foreign agencies.”

I know this, it’s obvious, and the authors of the report know it too, but in times of abundant information and opinions, which circulate swiftly and are used freely, all care with what is written and said is important. Especially with science.

The item titled “Collaboration and commercialization” uses three proxies: co-authored articles with industry, the number of patents, and citations of articles in patents. No data on commercialization is presented. We don’t know how we are doing in this area of research.

Here is a concrete example of how the source of data can change the analysis result. By WoS standards, Brazil would have 1.5% of articles co-authored with companies between 2019 and 2023. A quick search (less than five minutes) in a competing database shows 2.2% for the same period. The difference, considering the range presented in Figure 12 of the report, is significant and would change the country’s position, as well as global comparative conclusions.

The regional and state analyses shown in the report bring few surprises. Besides the usual suspects, states outside the traditional geographic axis appear with indicators that draw attention. For example, the State of Amazonas ranks highly in indicators such as international collaboration and articles in the top 10% of citations.

New question: what does this mean? Certainly, that there are good researchers in that state – as there are in all. But what else do we need to understand in order to design better-informed policies for the Amazon? There are many programs today focused on the Amazon that could benefit from solid evidence produced by research and research on research.

To close, the report shows a map of Brazil with the ten universities with the highest output in environmental sciences[6]. Interesting! It would be illustrative to know, in addition to the total number of publications per university, the number relative to the size of the university (for example, per faculty/researcher).

The report, therefore, brings indicators and analyses that are worth more for the questions they raise than for what can be conclusively drawn from them.

Bibliometrics is an important, useful, and powerful tool, and for this reason, it must be used with care to avoid inducing hasty analyses and conclusions that could damage the country’s science.

Capes, which has rightly been changing its evaluation model for postgraduate programs, incorporating quality indicators beyond the number of publications, citations, and international collaborations, in line with what is happening globally, certainly understands the limits and scope of the report.

As a national reference for Brazilian research, it is important that along with the report, a warning is included that much more needs to be done to create a full portrait – and a comprehensive overview – of Brazilian research. After all, everything that Capes does, supports, and disseminates has repercussions and signals values for research in the country, right?

Our own ability to design and analyze the state of Brazilian research can and should go further.

We continue in search of a contemporary overview and, above all, a clear picture.

[1] https://www.abcd.usp.br/wp-content/uploads/2024/08/Relatorio_panorama_da_pesquisa_brasil_clarivate-capes-agosto-2024.pdf

[2] For those wishing to examine comparisons between scientific databases: https://arxiv.org/html/2401.16359v1 and https://openalex.org/about. For journal coverage: https://link.springer.com/article/10.1007/s11192-021-03948-5 and https://link.springer.com/article/10.1007/s11192-024-04948-x?fromPaywallRec=false

[3] The report also presents research topics in which Brazil stands out, in Tables 11 and 12. Ten topics are presented in each, combining all fields of knowledge. It is noted there that “Zika Virus, Liquid-Liquid Phase Separation,” “Galaxy: Kinematics and Dynamics, Pyroptosis,” “Classification, Corporate Social Responsibility,” and “Forest Restoration, Knowledge Graph” are topics in which Brazilian research has a significant global impact. Well, but what does that mean?

[4] Saudi Arabia, Switzerland, Netherlands, United Kingdom, France, Australia, Canada, Germany, Spain, Italy, United States, Poland, Brazil, Japan, South Korea, Iran, Russia, India, Turkey, and China.

[5] See, for example: https://www.science.org/content/article/open-access-papers-draw-more-citations-broader-readership.

[6] An important topic, no doubt, but what topic isn’t?

Posted in Articles, Chapters and Books, Comunication