Thematic Seminar titled ‘Multilingualism in the Evaluation and Allocation of Resources’, conduced by André Brasil, researcher at CWTS and collaborator at CAPES.
The seminar focused on the following topics: within the research context, multilingualism emerges as a pillar of inclusivity and diversity gaining prominence on the global stage, even though it may not always receive equal recognition in Brazil. The Coalition for Advancement of Research Assessment (CoARA) emphasizes the importance of recognizing the diversity of scientific production, regardless of language. This perspective aligns with global initiatives such as the Helsinki Initiative and UNESCO recommendations, advocating for linguistic equity in scientific contributions. Considering publishing practices in Brazil and the availability of local open-access publishing platforms, it is essential for the country to continue valuing production in the local language despite ongoing pressures for internationalization. From an evaluation standpoint, addressing linguistic biases in metrics and classifications is crucial for a comprehensive and inclusive research ecosystem.
Thematic Seminar titled ‘How to Measure the Novelty of Research’ presented by Adriana Bin.
The seminar’s content was presented during the panel “Supporting Risky Research,” organized by the Serrapilheira Institute at the 75th Annual Meeting of SBPC in July 2023. Its objective was to provide an overview of what constitutes “risky research,” the challenges in funding such research, and its ex-post measurement. It offered a general perspective on the methodologies employed for this measurement based on selected works, along with the associated critical elements. Finally, some results from the evaluation of the 1st Public Science Call of the Serrapilheira Institute were presented.
Thematic Seminar titled ‘The Field of Research on Research and Innovation: Research Groups, Events, and Themes’. The Seminar was presented by different front members of the project, in a collaborative effort.
The objective of the seminar was to provide an overview of the state and context of the Research on Research and Innovation field globally, covering the following points:
(1) Report on the main events the team participated in;
(2) Key research groups and initiatives in the field worldwide;
(3) The future of the research agenda on the theme.
The Seminar aimed primarily to position our project within the field and contemplate future research possibilities, partnerships, and how we can contribute to the study of this field.
– Thematic Seminar titled ‘Identifying Themes in Large Volumes of Textual Data’, conducted by Daniela Maciel, Analyst at Embrapa and a Ph.D. candidate at DPCTO.
The presentation focus was on text mining, particularly two key stages of the process: tokenization and topic modeling. Tokenization, which involves breaking down a text into ‘tokens’ or individual words, is a starting point for many text mining methods. Topic modeling, on the other hand, is a technique used to discover ‘topics’ or hidden themes within a large volume of text. Two main sources were used: “Text Mining with R: A Tidy Approach” by Julia Silge and David Robinson, which provides an introduction to text mining in R, and the ‘topicmodels’ package by Bettina Grün and Kurt Hornik, a robust and flexible tool for fitting topic models in R.
– Thematic Seminar titled: ‘Assessment and Evaluation for Public Communication of Science and Technology’, conducted by Cibele Aguiar (Communication Front).
The seminar addressed the role of Public Communication of S&T and how federal public universities are dealing with the challenge of monitoring and evaluating their performance. The seminar’s objective was to present and discuss the issue of interest to scientists, communicators, institutions, and funding agencies: how to assess whether the S&T communication actions undertaken are leading to the achievement of objectives?
– A proposal for an E-book of the project was discussed.
– Working papers will be prepared by members of the fronts for publishing on preprint platforms.
Thematic Seminar titled: ‘Transformations in Brazilian Higher Education 2010-2020 and the Role of Research‘, conduced by members of Front 4.
The seminar addressed the following theme: Brazilian higher education has been diversifying, with different types of institutions playing different roles in the areas of research and teaching, and meeting the needs of different types of students, with various forms of organization and funding. To understand this process, Front 4 has been working on a typology of higher education institutions in the country, using data from INEP and CAPES in a historical series for the period 2010-2020. The seminar’s objective was to present and discuss some of the main results of this work so far.
Thematic Seminar titled: ‘Socioeconomic Analysis of Primary Industrial Property Data generated by INPI’, conducted by Prof. Sérgio Medeiros Paulino de Carvalho, Senior Specialist in Intellectual Property at INPI.
The seminar aimed to discuss impact indicators in Industrial Property for evaluating the impact of investments in science, technology, and innovation, as well as the use of IP data in Socioeconomic Analysis.
Thematic Seminar titled: ‘Things We Can Learn About Science, Technology, and Innovation in Brazil through the Analysis of Scientific Publications by Authors in the Country’, conduced by Prof. Carlos Henrique de Brito Cruz.
The seminar aimed to explore the characteristics and trends of the science, technology, and innovation system in Brazil through scientific publications authored by researchers based in the country.
– Thematic Seminar: ‘Exploring the Connection between FAPESP-Funded Research and its Policy Implications: Preliminary Findings using the Overton Database’, conduced by members of Front 1.
The seminar discussed the methodological approaches that Front 1 has been employing to identify the use of FAPESP-funded research in policy documents within the Overton database. Some preliminary research findings were presented, considering the dialogue with other (early-stage) studies using the same tool. Future steps and further analysis of the data were also discussed.
– Overview of the next steps for submitting the project to the Research Ethics Committee.
Thematic Seminar titled: ‘Alternatives to Peer Review in Funding Agencies’, conduced by members of Front 2.
The objective of the seminar was to present and discuss alternatives to peer review in funding agencies, such as randomization and the distribution and redistribution of funding within the scientific community. Additionally, the role of bibliometrics as a complementary or substitute method for peer review was discussed.
– The structure for the FAPESP report was approved by the Fronts.
– A pilot proposal for the typology of National Research Institutes was presented by Front 4.
– The first general meeting of the year was centered on outlining the activities scheduled for the first half of 2023. Key topics included deliberating the subsequent actions following the acquisition of primary research data, abstract submissions for upcoming events, and formulating plans for article submissions and other project activities.
– Discussion about the necessity of establishing a comprehensive calendar for the year 2023, including the allocation of specific dates for Thematic Seminars.
– Thematic Seminar titled: ‘Exploring Data for the Construction of Typologies of Higher Education Institutions and Science and Technology Institutions’, conduced by members of Front 4.
The seminar aimed to discuss strategies for collecting and consolidating data and information to construct typologies of Higher Education Institutions and Science and Technology Institutions. During the seminar, the team presented initial exploratory efforts undertaken to enhance the composition of relevant variables for typology creation.
– Following the seminar, the thematic and support fronts provided a comprehensive review of the activities completed to date. This included updates on the progress of literature reviews and the start of secondary data analysis. Furthermore, the discussion touched upon the future steps in the research process. The importance of obtaining primary data to advance the research activities was emphasized.
Thematic Seminar titled : ‘Effects of International Collaboration on Academic Careers’, conducded by members of Front 3.
The seminar provided an overview of the latest discussions regarding the effects of international collaboration on academic careers. The topic was examined from three perspectives: gender inequalities in international collaboration and two additional dimensions that are both influenced by and influence international collaboration, namely, mobility (national/international) and academic performance.
Thematic Seminar titled: ‘Data Management Model – Governance and Implementation’, conduced by member of the Database Support Front.
The presentation encompassed a comprehensive exploration of several key themes, including: conceptual framework, centralized databases, user experience, and key tools.
– Thematic Seminar titled: ‘Public Communication of S&T: Research and Actions of the Project’, conduced by members of the Communication Support Front.
The seminar had two main objectives: the project’s website presentation, and to foster a discussion about Science and Technology communication indicators. Additionally, the seminar presented insights into the research field of Public Communication of Science and Technology, highlighting its significance in the context of monitoring and evaluating CTI projects.
– Thematic Seminar titled: ‘An Overview of Statistical and Computational Methods with Potential Relevance to the Project’, conduced by members of the Data Analysis Support Front.
During this seminar, the team presented an array of current statistical and computational methods widely employed for analysis. The aim was to provide practical examples and introduce the fundamental concepts associated with each method. Some of the topics addressed included distinguishing between clustering and classification, exploring heuristic techniques, and delving into model-based approaches.
Moreover, members of the Data Analysis Team offered valuable insights by suggesting practical applications of these methods across all the Thematic Fronts.
– Thematic Seminar titled: ‘Approaches to Measure Research Impacts’, conduced by members of Front 1.
Its primary objective was to delve into the examination of research impact across three critical domains: social impact, economic impact, and public policies impact. The seminar encompassed in-depth discussions on theoretical and methodological perspectives, drawing from the recommended literature to identify, measure, and analyze the impacts of research.
– Thematic Seminar titled: ‘Towards a Typology of Higher Education and Research Institutions: Achievements and Next Steps’, conduced by members of Front 4.
The seminar had a three-part structure. Firstly, it provided a justification for the development of an institutional typology in higher education and research as a means to understand the diverse array of institutions and their distinct contributions. Secondly, it presented the outcomes achieved thus far. Lastly, it outlined the forthcoming stages, including the integration of new data and the expected results.
Thematic Seminar titled: ‘Mobility and Professional Trajectories’, organized by members of Front 2.
The seminar objective was to address the various impacts documented in the literature concerning national and international mobility in academic careers. The seminar also delved into the methodological pathways employed to assess these impacts, encompassing bibliometric, curriculum and academic identifier analysis.
Thematic Seminar titled: ‘Peer Review: Biases, Critiques, and Emerging Approaches’, organized by members of Front 2.
The primary objective of the thematic seminar was to engage in a discussion about the extensive use of peer review as a widely recognized practice within the scientific community for prioritizing research investments and publication choices. The seminar explored diverse applications of peer review in both ex-ante and ex-post evaluations, addressing the biases and critiques associated with its implementation. Additionally, the seminar introduced different variations and emerging applications of peer review.
– Thematic Seminar titled: ‘Excellence in Research: Evolution of the Concept, Approaches, Indicators, and Metrics’, organized by members of Front 1.
The objective of the thematic seminar was to discuss the evolution of the concepts of research excellence (along with practices, metrics, and indicators) and their implications for the four Research Fronts.
– Verification and confirmation of the databases presented during the previous general meeting, emphasizing the significance of select databases.
– Introduction of a spreadsheet outlining a proposal for Thematic Seminars. The inaugural seminar will be coordinated by Front 1 and will center on the theme of Research Excellence, a subject that binds together all project Research Fronts.
– Presentation of the spreadsheet that consolidates the databases listed and qualified by the team.
– Project management has initiated negotiations with the Unicamp library for the acquisition of relevant databases (Dimensions and Overton).
– Presentation of a consolidated analysis of the intersections of research questions to observe overlaps and complementarities.
– The initiation of the project’s Thematic Seminars was discussed, a crucial step for the Research Fronts to begin actively engaging in research activities (literature review, empirical research, etc.).
– Introduction of the FAPESP Indicators Management Team.
– Presentation of the work carried out by the FAPESP Indicators Management Team to align with the project’s database needs, data sources, and information requirements.
– Additionally, there was a discussion regarding the need for detailed Work Plans for the Thematic Fronts, particularly focusing on methodology and databases. The Thematic Fronts presented their research questions, which were thoroughly discussed by the participants.
– It was also decided that the Database Front would propose a consolidation plan for the databases and data sources identified throughout the previous steps.
– Presentation of a Work Plan by the Database Front.
– The Thematic Fronts have coordinated efforts to identify essential databases necessary for the execution of their research.
– Additionally, a presentation was made regarding the Work Plan for the Data Analysis Front, highlighting its potential statistical and econometric capabilities to support the Thematic Fronts.
– Clarification and refinement of the project’s organizational and governance structure.
– Elaboration and presentation of the detailed work plans for the 4 Thematic Fronts, as prepared by the respective coordinators.
– Establishment of a roadmap for the upcoming phases of the research.
– In this meeting, the strategy for managing project content and communication was also outlined.