Blog

Understanding the role of observed vs. expected observational studies in vaccine safety

In vaccine safety monitoring, observed vs. expected observational studies are used to assess the risk of rare and potentially serious adverse events following  vaccination. These studies compare the rate of disease or other health outcomes in vaccinated people with what is expected based on population rates or epidemiological data established before the vaccine was available (background rates). This type of analysis can identify clustering of adverse events and potential vaccine safety signals that could indicate new or unrecognised risks associated with receipt of a vaccine. Through use of these observational studies, experts can also better understand how vaccines may be contributing to population health outcomes.

While observed vs. expected observational studies contribute valuable information and evidence to assess the risk of rare or serious adverse events associated with immunisation they do have limitations.

Identification of potential signals that could indicate a new or unrecognised risk does not provide definitive answers about the safety of a vaccine.

• Observed vs. expected studies can only determine if there is a temporal association between receiving a vaccine and experiencing an adverse event.
• These studies cannot prove whether an adverse event is directly caused by a vaccination or whether it is due to other factors, such as an underlying health condition. 

Observed vs. expected studies should only be used in combination with other epidemiological evidence when assessing vaccine safety.

Examples of detection of safety signals using observed over expected studies

Myocarditis and pericarditis

In April 2021, the Israel Health Ministry advised that reports of heart muscle inflammation (myocarditis) after receiving a COVID-19 required further investigation. Regulatory agencies around the world responded immediately, checking adverse event reporting systems for reports of the condition after COVID-19 vaccination. Health professionals also responded, providing case reports in peer reviewed journals to ensure colleagues globally were aware that some people presented with myocarditis, and inflammation of the thin sac covering the heart (pericarditis) after receiving the SARS-CoV-2 mRNA vaccine.

Studies from the U.S. (2021), Canada (2022), Europe and Singapore (2023) provide examples of observed vs. expected observational studies that identified a potential safety signal related to the onset of myocarditis or pericarditis following COVID-19 vaccination. Subsequent studies, for example these studies from four European countries (2022) and Singapore and Hong Kong (2023) showed that there is an increased risk for some people following the receipt of COVID-19 mRNA vaccines, particularly after the second dose.

Watch a video about myocarditis, pericarditis and COVID 19 vaccines or read more about myocarditis and vaccines here.

Guillain-Barré syndrome

In July 2021, the EMA reported that there had been 108 cases of Guillain-Barré syndrome (GBS) reported from 21 million people who had received a SARS-CoV-2 viral vector vaccination. GBS is a syndrome that includes increasing muscle weakness or paralysis that may also have changed sensation (feeling). Regulatory agencies responded with further investigation of adverse event reports, as did health professionals with publishing case reports to inform colleagues globally.

Studies from the U.S. (2022), Australia (2022) and the U.S. (2023) provide examples of observed vs. expected observational studies that identified a temporal association between the receipt of a COVID-19 viral vector vaccine and the onset of GBS.
The occurrence of GBS after receiving a viral vector COVID-19 vaccine is very rare. Subsequent epidemiological studies, for example these studies from England (2022), Malaysia (2022), and Italy (2023), suggest an association between one of the two viral vector vaccines available.

Read more about Guillain-Barré syndrome and vaccines here.

What does this mean? Are vaccines still safe?

The COVID-19 vaccines that are in wide use are very safe. That does not mean that there are not side effects and potential risks. Most side effects are minor and don't last long. However, it is important to know what the risks are so that decisions about getting vaccinated are based on good scientific information.

For many people, getting COVID-19 disease has risks, such as getting very ill and needing to be admitted to hospital or developing serious health complications including disease-related myocarditis and/or pericarditis or Guillain-Barré syndrome. Long COVID is also a consideration. For most people, being immunised against the virus significantly reduces the risks related to having the disease.

Comparison of risk (benefit) with risk can be used to help with making decisions about any medicine, vaccine, or treatment.

Key messages

• Observed versus expected studies provide valuable information for identifying the risk of rare or serious adverse events associated with vaccination.
• Myocarditis and pericarditis have been identified as potential risk following mRNA vaccines and, thrombosis with thrombocytopenia, and Guillain-Barré syndrome have been identified as potential risks associated with viral vector vaccines through these observational studies.
• While O/E studies cannot definitively prove whether an adverse event is caused by a vaccine or not, they can provide valuable insight into the safety of a vaccine when used in combination with other evidence.
• The COVID-19 vaccines that are currently in wide use are very safe and the risks associated with them are much lower than those associated with the disease itself. It is important to be informed about all potential risks before making any decisions about getting vaccinated.