Throughout the pandemic, some in the public have acted in opposition to data-driven efforts such as vaccination and masking. The COVID Behaviors Dashboard presents a rich dataset on human behavior and beliefs around the globe that guides governments’ and health departments’ strategies to effectively mitigate COVID-19 in their areas.
The COVID Behaviors Dashboard is an initiative to better understand human behavior and decision-making surrounding the COVID-19 pandemic spearheaded by the Johns Hopkins Center for Communication Programs (CCP) in collaboration with Carnegie Mellon University, the University of Maryland, and Facebook. The dashboard contains data from over 12 million individual survey responses, providing insight into trends on masking, vaccine hesitancy, and trust around the globe. Director of Digital Strategy, Marla Shaivitz (MS), and Director of Monitoring, Evaluation, and Learning, Dr. Dominick Shattuck (DS), walk through the complexities of working with so much detailed data and the importance of subjective data to inform policymaker and health official responses to this pandemic and future health crises.
DS: The data presented in the COVID Behaviors Dashboard is intended for health officials who are leading COVID-19 health campaigns. They provide a high-level understanding of the trends in individual beliefs related to COVID-19 prevention and vaccine uptake, which helps health officials identify areas where they want to allocate resources to either deliver messages or conduct more research. We see these data as providing countries with a first and second step toward message dissemination.
MS: This data captures knowledge: what people perceive to be true and self-reported behaviors. We see in the dashboard that oftentimes people perceive others not engaging in a behavior, when in fact they self-report they are. In our work, questions that reflect this inconsistency focus on norms around specific behaviors. Take masking for example. In the United States from September 1-15, 26% of survey respondents believed that all or most people were wearing masks; however, 63% reported wearing a mask all or most of the time. Messaging can be developed that simultaneously encourages the community to increase its mask adherence, while also reinforcing the prevention behaviors of those individuals who are adhering to mask wearing. Normative theory suggests that individuals are more likely to enact a behavior if they believe a majority of people in their community are also enacting that behavior. Understanding which types of messages to develop, test and disseminate is imperative to reducing infection.
DS: The data in the dashboard are self-reported experiences and attitudes collected by our collaborators at CMU and UMD. Clinical data that you’re referring to provides important but different insights. It’s the combination of information from health facilities and self-reported experiences that should drive health messaging. Knowing levels of vaccine uptake in a community is important, but that piece of information does not provide insights on why individuals or sub-groups within a community are not showing up for vaccination. Self-reported experiences and attitudes provide those insights about what can be done outside of those procedural adjustments to service delivery and can inform other types of intervention, like messaging.
Participants were recruited through an advertisement delivered into their feeds on the Facebook platform. After clicking on that advertisement, they were sent to secure, remote servers managed by CMU and UMD. No individual information was shared back with Facebook. Like most studies, there are limitations to this dataset. The largest challenge is that participants are restricted to Facebook users, people with access to technology. This often leaves out individuals who are hardest to reach.
Although this is an extremely rich dataset that can play a key role in the development of health promotional activities, one database is not going to be enough to address all of the challenges presented by this pandemic.
DS: First, we analyze these data within the dashboard by presenting descriptive statistics for our target audience. Next, we conduct and present global or regional analyses that are presented in webinars and technical briefs on our dashboard and website. Analysis between countries is challenging because this sample does not ensure that each country is contributing the same proportion of participants as others. Finally, we conduct various local analyses for stakeholders such as local health departments and funders like USAID as they prepare for vaccine dissemination globally.
Analyzing topics like vaccine acceptance or hesitation is challenging in this current moment because every day, vaccine rollout expands and the data reflects changes in acceptance with the presence of vaccines. Based on our initial dashboards and analyses, acceptance increases following shortly after rollout begins. It could be that the actual presence of vaccines in communities changes attitudes, as the vaccination becomes less abstract.
MS: It has been important to consult the user personas we developed early on in this process to make sure our decision-making was aligned with the needs of our audience – policy makers, public health officials at the WHO, and public health communication professionals. We have been purposeful in crafting a data story, to give these audiences a fuller picture of knowledge and beliefs around COVID-19 in their countries and regionally. A challenge has been acquiring a sample size large enough in some countries to be representative, as well as robust enough within demographic categories. We see there aren’t enough women taking the survey for us to display the results in some countries.
The conclusions come in four areas: the benefits of interactive design, the importance of trust in communication, the desire for information, and global health outcomes.
MS: In telling the data story, the ability to home in on a particular country’s data longitudinally has benefitted countries, as has the demographic breakdown – this is the “who”. We framed each visualization with a question, such as ‘Among unvaccinated participants, who is most willing to accept a vaccine?’ The interactivity demonstrates who the largest group is - by gender, residence, age and education level. This helps public health professionals focus messaging on potential acceptors.
It has also been interesting to see throughout this project how trust in information sources varies from country to country. The survey asks, ‘Which sources of COVID-19 information were most trusted and most frequently accessed in different countries?’ This is important to identify trusted messengers. Globally, politicians do not fare well as trusted messengers. According to the data, generally people do tend to trust health experts and scientists.
DS: Our team presents these data to national and regional COVID-19 technical working groups, tailoring the presentations to what is most relevant for many low- and middle-income countries. These countries have a significant data deficit, which limits their ability to make informed decisions about vaccine rollout and prevention messaging. It definitely reinforced my understanding of the data gaps when we began doing this work more than a year ago.
Another major conclusion I’ve arrived at looking at these data is that unvaccinated individuals are expressing their concern around three specific elements: side-effects, safety, and efficacy. In order to be effective, messaging will need to address these concerns in ways that reach particular sub-audiences. The WHO quickly labelled the conspiracy theories about these elements part of the larger COVID-19 ‘infodemic’ in early 2020. Over the last year, we have seen how this infodemic has sowed the seeds of doubt in individuals around the world, while at the same time the scientific community has continued to build evidence supporting the safety and efficacy of vaccines. It’s time to find ways to articulate those results in ways that are most salient and using channels that are most trusted by unvaccinated individuals.
MS: That’s one of the aims of the dashboard, to give those who need to plan out budgets and campaigns the information needed to allocate resources. How large is the percentage of those who say they will definitely not accept a vaccine versus those who will probably accept? What is the demographic breakdown of each of those groups? This helps policymakers and public health professionals focus efforts for greatest impact.
DS: Prioritization is challenging in the United States, where we have so much information available and lots of resources, compared with other countries. We know that concerns about vaccine safety and efficacy are main reasons for non-vaccination. Yet, the data don’t support those concerns.
Not every individual consumes and assesses the validity of health messaging in the same way. Factors such as the source of information, phraseology of the message, and images associated with the messaging speak to different people differently. Social media platforms have been implementing this approach for a decade and keeping us engaged. Tailoring messages for sub-audiences will be critical for moving the needle on vaccine uptake.