Will a COVID-19 Vaccine Benefit You? (Or Me?)

Key Points:

• The primary endpoint of the Operation Warp Speed vaccine trials is a reduction of coronavirus disease, which includes mild COVID-19
• Mild COVID-19 was deliberately included as a primary endpoint, in part because vaccine trial participants are likely to be diagnosed early in their illness
• Disease severity is measured over several weeks for every trial participant who develops COVID-19 and will be an important consideration when assessing vaccine efficacy

There has been much discourse in the lay and medical press that, in the COVID-19 Operation Warp Speed (OWS) trials, vaccine efficacy includes consideration of the “trivial” characteristics of mild COVID-19 disease and is not exclusively focused on the medically important complications of COVID-19 disease, hospitalization, and death. When one reads the trial protocols,^{1 2 3 4} one sees that the primary endpoint of the OWS vaccine trials is a reduction of coronavirus disease, which includes reducing the signs and symptoms of mild COVID-19 illness (e.g., headache, cough, fever, myalgias, signs of loss of taste and smell).

By including mild COVID-19 disease within a primary endpoint, the trials can evaluate the vast disease spectrum of COVID-19. This approach will ultimately allow for widespread licensure and efficient real-world rapid assessment of the vaccines’ effect on COVID-19. For companies developing vaccines, inclusion of mild COVID-19 disease within a primary endpoint for the OWS trials makes sense; these companies recognize the wide range of COVID-19 disease, from mild to severe, and want to produce an effective vaccine with widespread benefit.

Importantly, reading further into the vaccine trial protocols reveals a deeper evaluation of disease severity within each of the OWS studies: the trials have an endpoint that includes measurement of disease severity at diagnosis and over three-weeks of follow-up for every participant who develops COVID-19. If one looks in the tables at the end of the protocols, (Table 17 in the Moderna protocol,⁵ Table 4 in the AstraZeneca protocol,⁶ and Appendices 2, 6-8 in the Janssen protocol⁷), one sees that every person in the trials with COVID-19 infection is intensively followed from the time of diagnosis over the next three weeks with standardized assessments of their signs and symptoms, including the use of medical facilities, hospitalizations, and what as investigators we all hope is rare – death.

At the time the vaccine trials were designed, there was no consensus about the frequency with which those who were willing to enroll would acquire COVID-19 and, more importantly, what the spectrum of COVID-19 disease among trial participants would be. In fact, we were pretty certain we would not see what, at that point, had been the norm in the pandemic (i.e., large numbers of people presenting to hospital emergency rooms with full-blown end-stage pneumonia). We suspected that in the trials we would see almost the opposite of this – participants presenting earlier and with milder COVID-19 symptoms than what was, at that time, being seen in medical practice or in any published cohorts of persons.

Persons who enroll in a clinical trial, especially of the complexity seen in the COVID-19 vaccine efficacy trials, generally have to be interested and concerned about the disease. And, the design of the OWS trials involves active disease surveillance and no barrier to COVID-19 testing for 30,000 participants per study. Moreover, these 30,000 participants are knowledgeable about COVID-19, likely anxious about getting COVID-19, and the trials make them attuned to any aspect of the disease, all the time. Participants in the trial receive weekly calls/texts about coming to the clinic at the first signs of any respiratory illness or fever and, importantly, trial participants have essentially unlimited access to COVID-19 testing. Thus, we felt that classification of COVID-19 disease severity at the time of diagnosis would not really provide an accurate reflection of a participant’s COVID-19 illness. We expected that most trial participants would get worse after an initial diagnosis of COVID-19 and, as such, the real endpoint evaluation of disease severity – and hence the ultimate “primary endpoint” – would be the constellation of signs and symptoms that were collected over the two to three weeks post-diagnosis. The section outlining the detailed follow-up of COVID-19 cases within each study had its own schematic diagrams/tables and was “costed out” for site funding separately.

One of the harmonizing aspects across all the vaccine trials is that participants with COVID-19 are queried daily with a standardized questionnaire evaluating their signs and symptoms on a mild-to-moderate-to-severe scale. Samples for viral shedding in saliva or the nose are taken every second to fourth day, depending on the protocol, to evaluate the duration for which the virus is shed. Duration of fever and pulse oximetry data are also recorded. These data allow one to grade the rate of progression, duration of symptoms, duration of signs, duration and severity of systemic effects, and need for any follow-up medical interventions (e.g., physician visits, telemedicine visits, ER visits, hospitalizations, and any complications of hospitalizations). Evaluations of these data are blinded and will be analyzed between the vaccinated and placebo recipients to determine whether disease duration and severity are reduced.

Data outlined in the prior paragraph are listed as secondary endpoints in the trials and while “relegated” to this section, it does not mean their importance is secondary. This was highlighted in a recent paper on using burden of disease as the primary endpoint in COVID-19 vaccine studies by Mehrotra and colleagues,⁸ as well as recent analyses of data from the monoclonal antibody studies conducted by Lilly and Regeneron.⁹ The monoclonal antibody studies do a nice job of aggregating medically complicated disease to show the benefits of early use of the antibodies. They also show one approach to compiling recorded signs and symptoms of COVID-19 to show clinical benefit to patients over time. This offers a model to structure analyses both within and between individual and collated vaccine studies. In addition, the prospective close follow-up that will occur in the vaccine studies lays the groundwork for future studies and maximizes what we learn from the 150 expected COVID-19 cases in each vaccine trial.

It is unlikely that, in any individual trial, we will see enough hospitalization and death to evaluate efficacy of a vaccine versus placebo for severe COVID-19 disease. Obtaining this information within a trial would require lengthening the time to determine whether a vaccine works by 18 months –something none of us want to see. Vaccines for other diseases offer a useful perspective about the inclusion of mild illness as a primary endpoint within a trial: data from other vaccines show that those capable of modifying outpatient/modest illnesses have almost universally been capable of modifying more severe illness. With most vaccines, efficacy in reducing the severe spectrum of a disease is actually somewhat higher than for mild aspects of a disease. Let us hope that vaccination will easily produce discernible, measurable, medically important differences in COVID-19 disease in the populations enrolled in the trials. The current surge in activity in the U.S. clearly shows that we need vaccines to combat this epidemic.

Author’s postscript: After writing this piece, but before its posted publication, the press release of a 90% effective rate among persons with symptomatic disease was announced by Pfizer on Monday, November 9, 2020. While specific data are not available, attaining this milestone is a triumph for science and provides optimism that COVID-19 vaccines will be a major tool for curbing the medical, economic, and social aspects of this pandemic.

Dr. Larry Corey is an internationally renowned expert in virology, immunology, and vaccine development, and a major adviser to OWS. He leads the COVID-19 Prevention Network (CoVPN), which was formed by the National Institute of Allergy and Infectious Diseases at the U.S. National Institutes of Health to respond to the global pandemic. He will be a regular contributor to this Johns Hopkins University-University of Washington Blog series.

Additional References:

“Will covid-19 vaccines save lives? Current trials aren’t designed to tell us.” BMJ 2020;371:m4037; http://dx.doi.org/10.1136/bmj.m4037. Published: 21 October 2020.

Eric J. Topol, MD; Paul A. Offit, MD. “Paul Offit's Biggest Concern About COVID Vaccines.” Medscape. https://www.medscape.com/viewarticle/936937. Published: 09 September 2020.

¹ Moderna TX. Protocol mRNA-1273-P301, Amendment 3. 2020. https://www.modernatx.com/sites/default/files/mRNA-1273-P301-Protocol.pdf.

² Pfizer. PF-07302048 (BNT162 RNA-Based COVID-19 Vaccines) Protocol C4591001. 2020. https://pfe-pfizercom-d8-prod.s3.amazonaws.com/2020-09/C4591001_Clinical_Protocol.pdf.

³ AstraZeneca. Clinical Study Protocol - Amendment 2 AZD1222- D8110C00001. 2020. https://s3.amazonaws.com/ctr-med-7111/D8110C00001/52bec400-80f6-4c1b-8791-0483923d0867/c8070a4e-6a9d-46f9-8c32-cece903592b9/D8110C00001_CSP-v2.pdf.

⁴ Janssen Vaccines & Prevention BV. VAC31518 (JNJ-78436735) clinical protocol VAC31518COV3001 amendment 1. 2020. https://www.jnj.com/coronavirus/covid-19-phase-3-study-clinical-protocol.

⁵ Moderna TX. Protocol mRNA-1273-P301, Amendment 3. 2020. https://www.modernatx.com/sites/default/files/mRNA-1273-P301-Protocol.pdf.

⁶ AstraZeneca. Clinical Study Protocol - Amendment 2 AZD1222- D8110C00001. 2020. https://s3.amazonaws.com/ctr-med-7111/D8110C00001/52bec400-80f6-4c1b-8791-0483923d0867/c8070a4e-6a9d-46f9-8c32-cece903592b9/D8110C00001_CSP-v2.pdf.

⁷ Janssen Vaccines & Prevention BV. VAC31518 (JNJ-78436735) clinical protocol VAC31518COV3001 amendment 1. 2020. https://www.jnj.com/coronavirus/covid-19-phase-3-study-clinical-protocol.

⁸ Mehrotra, DV, et al. “Clinical Endpoints for Evaluating Efficacy in COVID-19 Vaccine Trials.” Annals of Internal Medicine. https://doi.org/10.7326/M20-6169. Published: 22 October 2020(https://doi.org/10.7326/M20-6169. Published: 22 October 2020).

⁹ Chen, Peter, et al. “SARS-CoV-2 Neutralizing Antibody LY-CoV555 in Outpatients with Covid-19.” The New England Journal of Medicine. https://www.nejm.org/doi/full/10.1056/NEJMoa2029849. Published: 28 October 2020(https://www.nejm.org/doi/full/10.1056/NEJMoa2029849. Published: 28 October 2020).