Informing policy with science
Introduction
As individuals, every day we make decisions, and in doing so, we often wish we had a crystal ball to predict the consequences of our choices. As a society, we also make decisions, most frequently through elected representatives, and we desire to foresee the consequences of our actions, especially when addressing societal challenges such as climate change, cybersecurity, fake news, and pandemic diseases, among others. Research can predict outcomes based on data, and through the scientific method, biases can be minimized. Therefore, research can be beneficial for society by advising its representatives and implementers, and by providing them with such a powerful predictive tool.
Although the connection between social representatives, policymakers, or other decision-makers, and researchers may seem straightforward in the above paragraph, this connection does not always occur smoothly. Several factors contribute to hindering this connection: policy and research are different worlds, with their own pace, interests, methods, rewards, and language of communication diverging in many aspects. Moreover, researchers tend to specialize in small niches in order to provide high-quality data; however, this specialization poses a difficulty in maintaining a holistic vision of a topic, which is necessary to advise policymakers effectively. On the other hand, policymakers require quick answers and a level of certainty that research cannot always provide.Therefore, it is necessary to have a connecting figure who understands the methods of research, its language, and its limitations, while simultaneously maintaining a big-picture approach to a topic by connecting different dots of research data to inform predictions of consequences and future scenarios related to broad societal challenges. These individuals are known as research evidence officers or science for policy officers. To assist policymakers in the decision-making process, research evidence officers provide them with balanced and relevant information gathered through the scientific method by the research community.
Different decision-making institutions, such as national or regional parliaments, judicial courts, or other government agencies, have various bodies or offices of research evidence officers. These bodies or offices organize the communication with the organizations they advise, as well as the communication with the research community, in diverse ways. Depending on the balance between channelling communication from the policy makers, coordinating the research community inputs, and reporting to the advised organization, the role of the research evidence officer varies. Sometimes, they can also function as a channel of communication with associations or societies of researchers in addition to reporting directly to the advised body. However, a commonality among most of them is the final outcome they produce: a report based on research evidence that directly informs policymakers.
An important aspect of research evidence officers is that they are rarely experts in the topic about which they are informing the policy bodies. Often, the topic is not within the discipline they were originally academically trained for. However, this is not crucial; they utilize their skills and processes to work with the scientific community and sort which evidence is relevant and of good quality. They are experts in searching, selecting, and presenting information gathered through research to a very specific audience: societal decision-makers.
Practice case
Are you ready to experience some of the tasks that a Research Evidence Officer perform? Get ready for the Research Evidence Officer REBECA Practice case!! Remember, this practice case does not prepare you to become a Research Evidence Officer, only it aids you to better decide if this is your type of profession. After completing the case, please do the reflection exercises; it will help you to clarify what you have experienced and take decisions.
Requisites to perform this task: None
After a meeting with your team, you have been restricted to choose between these 3 topics to write the report. These are the parliament selected topics:
• “The impact of child obesity in public health”,
• “Artificial Intelligence as a tool to take decisions in public administrations”,
• “Migration due to climate change and the impact on immigration policies”.
Action: Please, select one topic from the given list to continue with the practice case.
NOTE: different offices have different methods to assign the topics to research evidence officers. To give you more options, we have provided you with the opportunity to select one. Most times, this is not the case, and the topic is assigned to you.
Before starting to read research evidence data, it is important to understand why the issues and perspectives of this topic are relevant to society. These will be the key points that you will need to address in the report.
There are two major sources to start looking: general media and socioeconomic magazines, and policy or documents generated by umbrella decision making bodies, or global or sectorial organizations, like European Commission, European Council, United Nations, Intergovernmental Organization of Migration, the World Health Organization, The Intergovernmental Panel on Climate Change, Carnegie Endowment for International Peace, and the Organization for Economic Cooperation and Development to name a few.
Action:using relevant social non-research sources make a list of 3-4 points that you found the most relevant to deal with after reading these articles.
Other advisory offices in the world have may covered this topic directly or indirectly. These sources can be very valuable to you to double check your selected key points and their relevance.
Action: search in the repositories of the Joint Research Council (JRC), the Office of Technology Assessment at the German Bundestag (TAB), the Parliamentary Office of Science and Technology in the UK (POST) and find if they have informed their parliaments about this topic or similar. Then quickly check the summaries of their reports and compare their key points with the ones you have pinpoint, and if you have missed any important topic.
It may happen that this topic has already been discussed in a session of the parliament of your country. It is interesting to understand how the decision makers of your country dealt with the topic, what they found more interesting, where there was a clear gap of knowledge, or which was the more controversial point. This will help you to further tune your report and make it more relevant and useful for policymakers.
Optional action: if you have access to the sessions of the parliament of your country, check if the topic has been dealt with and make a list of gaps of knowledge and controversial points. Alternatively, you can search in the sessions of the EU Parliament .
NOTE: these tasks may be assisted by clerks of parliament whose one of the tasks is providing you these documents
Now, you understand what is relevant for society, and how your main audience think about it, it is time to check what is the opinion of the research community, how they tackle this issue, and what scientific evidence exists or where are the gaps.
Action: search in the research literature the references dealing with your topic. Refine the research only using peer-review articles. Check in a couple of reviews, if the research evidence so far found in your selected topic is dealing with some of the subtopics that you have found relevant for society.
NOTE: Research evidence officers pay a lot attention to the validity of the published studies that they use to gather information. Peer review, unbiased studies, lack of conflict of interest, and good-quality experiments are key aspects to check before considering information appropriate.
Written material is not the only source of research evidence information. On many occasions, important information regarding the big picture of a topic, tacit knowledge within the research community or dissented issues do not appear in research articles. The same is true for nuances or incomplete pieces of evidence or research gaps. For this reason, researchers are directly needed to be asked. Engaging with experts can also bring about angles of the issue that may have escaped to your search, so expert interview is not a final step on your report outlining, rather a very early one.
Action: make a list of 3 researchers that after your bibliographic search you can consider experts on your selected topic. Your aim is to cover at least the main areas/subtopics that you have pinpoint in your previous steps.
TIP FROM THE EXPERT: Interviewing first experts with a broad vision of the topic is an asset. It is also important to cover different scientific perspectives ideally (social sciences, vs environmental science, vs hard sciences etc.). If you are focusing at country level, probably local experts should be considered first as they know best the local socio-politico-economic context.
NOTE: In many occasions, research evidence offices have advisory scientific boards or work closely to research societies, so even the selection of experts to interview is shared with research community. The number of interviewed researchers vary depending on the complexity of the topic and the controversy among researchers. A typical report of the Advisory office to the Spanish Parliament (Oficina C) has 15-20 interviewed experts.
The information, whether cited from a peer review paper or gathered from an interviewed expert, needs to be of good quality. After all, it is the research community voice what legitimizes your report as a piece of research evidence. For this reason, it is important to check if the source of information is of good quality. A very important issue is that your experts do not have conflict of interests.
Action: please find attached the information extracted from the Declaration of Interests of one of your experts. Check if your selected researchers comply with these rules. You may search through internet to verify this information.
NOTE: on many occasions, experts are not only researchers from the academia, but also from the industry, or even from advocacy groups who can bring different points of views to a topic, however interests have to be balanced so the advice is not biased.
After assuring that expert 1 comply with the rules of conflict of interests, and inviting this researcher to become interviewed, you need to prepare the list of questions that you would like to ask to this researcher.
Action:Prepare a list of questions to ask to the first expert.
NOTE: it could happen that the expert feels more inclined to answer in depth some questions than others or even consider some of your questions irrelevant because are not of interest to him/her. Interviewing skills are difficult skills who can be trained, so do not despair.
TIP FROM THE EXPERT: As you are gathering evidence on a topic, encourage the expert to tell you what is known (which is easier), what is unknown (research gaps) and unknowable (things we can never explore), so you can clarify the limitations of the scientific advice with the policy makers.
After your first search, and first interviews, you can outline the structure of the report. The report will contain an introduction that set the relevance of the topic, and the impact on society that you have pinpoint on task 1, 2 and 3. Then, sections will follow, each dealing with the subtopic or perspectives of the topic that you have also extracted in the first tasks.
Action: make the draft of the structure of the report
You have filled all the outlined sections- already tailored based on the relevance to society and policy makers- with all the information gathered in your bibliographic research and interviews. However, the language used makes the text to be understood by an expert in the topic, a researcher, not a policy maker. Your text needs the language to be refined and some concepts to be elaborated, so lay audience can understand it.
Action: depending on the topic selected, please refine the following definitions and explain the following concepts in 3 sentences for lay audience.
• The definition of AI for lay audience and explain the concept of machine learning in 3 sentences.
• The definition of obesity and explain the concept of comorbidity in 3 sentences.
• The definition of climate migrants and explain the concept of climate change in 3 sentences.
NOTE: It is also very important to double check to avoid direct recommendations. Remember, your role is to inform about the evidence based in research, not to intervene in the decision-making process. This is the degree of distance to the advised organization (Parliament of your country) that your advisory organization has established. In other organizations, this degree of distance may vary, and the intention of the advice is higher, for example, they can propose “policy options” or even recommendations.
Congratulations! You have finished the report, but this is not the final version. It will go for rounds of revisions: first, the experts will review and give you feedback, they may be not agreeing in some points, and you will need to find a consensus or highlighted the dissensus. Then, your colleges will go for rounds of review to check if there are contradictions, the messages are clear, the language is understandable.
TIPS FROM THE EXPERTS: Whenever you receive feedback in your manuscripts, it is always for the better: the improvement is assured.
Acknowledgements
This practice case has been created thanks to the input of three research evidence officers:
- Jose Luis Roscales, Research Evidence Officer at Oficina C.
- Maire Franquin, Scientific Policy Officer, Euro-Case..
- Steffen Albrecht, Senior researcher, Office of Technology Assessment at the German Bundestag
They also validated this case after its development.
Guided reflection
After this experience, we suggest you reflect on the following questions:
- Did you find the practice case easy or difficult to accomplish?
- What was the most engaging task for you? Was it difficult or easy?
- What was the most challenging task for you? Did you enjoy performing it? Would you see yourself getting better at it?
- Have you found something new about this profession? What was it? Did it surprise you? Did you like it or dislike it?
- Do you feel like contacting a research evidence officer in your network and research a little bit about the profession? Where would you find it?
Further information
If you want to know more on the profession, or specialised training information, check the:
- Information and resources for Evidence-Informed Policy making
- Competence Framework ‘Science for Policy’ for researchers
You can also follow some of our recommendations to explore careers beyond academia.