The Missing Gaps in Covid-19 Data: Reflections From Seven Country Case Studies

Technology Policy Covid-19

The Missing Gaps in Covid-19 Data: Reflections From Seven Country Case Studies

Commentary
Posted on: 15th February 2021
Hermione Dace
Policy Analyst

    Data is key to a national and international pandemic response. Understanding who has the virus, where and how it spreads, and the rollout and performance of different types of vaccines, therapeutics and testing are all critical to providing insight that will inform decision-making.

    Yet, the way countries are currently managing Covid-19 data is mostly inadequate. As a result, we recently argued for radical reform of our data infrastructure to beat Covid-19.

    As a follow-up, in this article we have chosen seven countries across the world to examine the way they are handling Covid-19 data. We identify where countries are falling short and consider where there is best practice. Taken as a whole, it’s clear there are significant gaps in the way countries record, aggregate, analyse and report data for Covid-19.

    How Are Countries Handling Covid-19 Data?

    How Are Countries Handling Covid-19 Data?

    Tables 1 and 2, below, assesses the countries across six indicators. It shows that the way most countries handle data for testing, tracking cases and vaccinations is complex and fragmented, with several organisations holding responsibility for data management. In the UK, for example, test and vaccination data is generated across a huge number of sites and responsibility is divided across multiple different organisations. In Germany, individual states have their own data systems and strategies for vaccine rollout.

    Most countries take a different approach to each other when recording and aggregating data. Both Israel and South Korea have relatively high-tech approaches to collecting data, whereas Japan and the US have been reported to use manual processes and fax machines, which leaves room for errors. Although most countries report their data through dashboards, the success of reporting varies. South Korea reports its data in near-real-time, but many other countries do not, making it difficult for researchers to provide policymakers with appropriate advice at the right time.

    Table 1 – Covid-19 data infrastructure comparison across the UK, Germany and the US

     

    United Kingdom

    Germany

    United States

    User journeys that generate data

    Hospitals, NHS locations and care homes, regional test sites, mobile unit, at home, GP surgeries, conference venues, sports centres etc.

    Mass vaccination centres, GPs, mobile sites/travelling vaccination teams. 

    Community testing sites, at home, drive through and walk in locations, pharmacies

    Parts of the system involved

    Depends on type of data and location

    Health and social care organisations: DHSC, PHE, NHSX, NHS Improvement, NHSX, NHS Trusts

    Multiple labs

     

    Local authorities, GPs, and hospitals

    State and local health departments, private healthcare providers, labs

    How is the data recorded?

    Mostly on local computer systems

     

    On computer systems (through the German Electronic Reporting and Information System for Infection Prevention), or sometimes manually on Excel

    Positive cases are uploaded on to a highly automated Epidemiological Investigation Support System

    How is the data aggregated?

    Data is uploaded to NHS database

    Excel has also been used

    Official records are held by local authorities across the country, with whom residents must register by law. State officials can then access these for specific purposes.

    Vaccine data is uploaded on to CDC’s software

    Many hospitals transfer data by hand and many health

    CDC is using Oracle’s National Electronic Health Records Cloud, to manage the distribution of vaccines departments share data by fax

     

     

    How is it reported/ analysed by decision-makers?

    Public dashboard: reports testing data which developers and analysts can access

    Vaccination data is available for the public to view on excel and pdf. online

    Public dashboard: maintained by The Robert Koch Institute

    Public dashboard: every state has at least one dashboard, but every dashboard is different

    Volunteer efforts are some of the best reporting mechanisms and are used by government

    Main failings/ shortfalls

    Data has been lost or incorrectly reported to the inadequacy of systems used

    Not enough data is collected

    Individual states are responsible for their own vaccine rollout plans, so data collection is fragmented and varies by region. 

    Unclear how this feeds into national figures based on limited publicly available info

    Particularly limited contact tracing and testing data across all states

    Very few public health indicators are reported on

     

    User journeys that generate data

    Hospitals, NHS locations and care homes, regional test sites, mobile unit, at home, GP surgeries, conference venues, sports centres etc.

    Mass vaccination centres, GPs, mobile sites/travelling vaccination teams. 

    Community testing sites, at home, drive through and walk in locations, pharmacies

     

    Table 2 – Covid-19 data infrastructure comparison across Japan, South Africa, Israel and South Korea

     

    Japan

    South Africa

    Israel

    South Korea

    User journeys that generate data

    Public health centres, large hospitals

    Hospitals, mass vaccination centres, and pharmacies

    HMOs, Homefront Command, Hospitals (public and private), airports

    Testing booths, drive-through and walk-through centres, hospitals, public health centres

    Parts of the system involved

    Local health centres, regional medical associations, Japan’s Institute for Infectious Diseases (NIID)

    GPs, hospitals, and pharmacies

    HMO clinics and complexes, spread widely, close to all population centres

    Homefront command complexes, all hospitals, mobile clinics for nursing homes, private hospitals/clinics and airport private labs

     

    A coordinated network of public health centres in 250 districts

    How is the data recorded?

    Predominantly doctors fill in a form by hand

    All people vaccinated will be on a national register and will receive a vaccination card

    Test data is recorded through a personal digital HMO card that enters the HMO-wide digital system

    Vaccination data is recorded by swiping one’s digital HMO ID card at the time of vaccination

    Contact tracing data is recorded on the Epidemiological Investigation Support System (EISS)

    How is the data aggregated?

    Data is faxed to local health departments, then to central govt who manually type it in and collate it on a computer system

    Unclear, data system is still being developed

    Digitally through HMO databases and sent to the MoH daily

    Public health centres send information rapidly to the Korea Centers for Disease Control and Prevention

    How is it reported/ analysed by decision-makers?

    Public dashboard: limited data is reported on separate online dashboards

    Public dashboard: is currently being developed and will show daily numbers

    Public dashboard: all data is aggregated from the 4 HMOs, state hospitals and Homefront command to the MoH in dashboards and briefings, and then numbers are published on the MoH public website

    Public dashboard: the Korea Centers for Disease Control and Prevention updates its website with near-real-tine information on local outbreaks

    The site reports several Covid-19 stats for every region of the country

    Main failings/ shortfalls

    Not enough data is collected

    Poor/outdated (manual) methods of data collection, aggregation, analysis

    Transparency of testing/disease data has been questioned

    The South African government is in the process of setting up a new data system that will be used during vaccine rollout, so public data on this system is limited.

     

    N/A

    User journeys that generate data

    Public health centres, large hospitals

    Hospitals, mass vaccination centres, and pharmacies

    HMOs, Homefront Command, Hospitals (public and private), airports

    Testing booths, drive-through and walk-through centres, hospitals, public health centres

     

    What Are The Missing Gaps?

    What Are The Missing Gaps?

    Although some countries have fared better than others at managing data throughout the pandemic, there are some common weaknesses across the case studies analysed. 

    Collecting and Recording Data

    There is not enough granular data being collected. Most countries are only collecting basic information, such as the number of positive tests, number of deaths and the number of people who have been vaccinated. In the US, for example, many health departments aren’t recording where a positive case caught the virus. Although countries are recording who has been vaccinated, most countries do not collect health data on race and ethnicity, gender identity, job category and socioeconomic status. Often, where this data is available, it is not properly disaggregated so cannot be useful in analysis. Most countries are also not collecting follow-up vaccination data on side-effects or symptoms or collecting data on who has turned down the vaccine. All this data is necessary for optimising the pandemic response.

    The methods used to record the data are outdated, leaving room for errors. In the UK, PHE previously admitted that 16,000 confirmed Covid-19 cases were missed from daily figures because of a problem on Excel. Very little of Japan’s testing or reporting system is automated. Reports of new covid-19 cases are submitted by doctors, who fill in a form by hand and then fax it to local health departments. In the US, many hospitals also transfer data by hand and many health departments share data by fax. Problems with a vaccine administration software (VAMS) has meant that vaccinations have been recorded on paper first. This means that there can be a delayed response as manual processes are generally slower. It also leaves more room for error. However, there are some ongoing efforts to correct this. For example, the CDC is using Oracle’s National Electronic Health Records Cloud, as well as Oracle’s Public Health Management Suite to manage the distribution of Covid-19 vaccines throughout the United States. It receives anonymised data from all US jurisdictions administering vaccines and will be used for reporting and analysis.

    Aggregating and Analysing Data

    Data collection and organisation is fragmented. Often, rather than there being a single body responsible for data, responsibility is spread across multiple organisations and departments. In Germany, individual states are responsible for their own vaccine rollout plans, so data collection is fragmented and varies by location. The software that states use to record information is often incompatible. In the US, most health systems, public health departments and jurisdictions have their own processes for managing data, meaning it’s very difficult to get an accurate and timely picture of the pandemic. For example, Covid-19 hospital admissions were measured by the NHSN, but cases coming to emergency departments were reported in a different database.

    There is a lack of common standards within and between countries. The US, for example, does not have national, state, county or city-level standards for Covid-19 data. There are no nationwide requirements for the information that hospitals and testing labs must report to health departments. In the UK, data on hospital admissions is not updated every day by all four nations and the figures for Wales are not comparable to other nations as they include suspected Covid-19 cases. Too often data doesn’t meet national standards and isn’t combined.

    Reporting Data

    Data is not reported in real-time. In Japan, for example, testing data from private labs is only added to the public dashboard weekly. This makes it very difficult for researchers, as they try to advise authorities how best to manage the virus and save lives. In the US, clinics in some states have been blamed for delays in their vaccine reporting, due to glitches in online systems.

    Data is reported inconsistently, or not at all. There have been multiple instances where data has been reported incorrectly or inconsistently, making it difficult to respond properly to the crisis. For example, the US Centers for Disease Control and Prevention (CDC) stated that 27 healthcare workers had died from Covid-19 in their first tally, when the figure was in fact much higher. UK testing data includes results from both pillar 1 and pillar 2 testing. Up until 1st July, these data were collected separately meaning that people who tested positive via both methods were counted twice. Conflicting data makes it very difficult to plan and distribute resources to the places most in need.

    Data is reported in a way that makes it difficult to use by others. Data needs to be available in a way that means academics, journalists and software developers can use it to add value. This means information must be in machine-readable format and based on open data standards. Yet, a lot of data remains inaccessible. In Japan, very little data on testing and case reporting is available at all, so outside researchers have no way to independently validate, replicate or scrutinise it. It’s been reported that researchers have to fill out several forms before being able to access data.

    Best Practice Data Infrastructure

    Best Practice Data Infrastructure

    South Korea's Organised Approach to Test and Trace

    Early in the pandemic, South Korea focused on rapid and widespread testing, and tracking of all contacts of those exposed. The country created a new, highly automated system called the Epidemiological Investigation Support System (EISS). When someone tests positive, their information is uploaded to the system. This is combined with credit card and smartphone data to see where a citizen has travelled. The data then quickly produces both close and distant contacts. South Korea’s success in data management correlates with its relative success in controlling the outbreak.

    Israel's Data-savvy HMOs are Key to its Vaccine Success

    Israel’s sophisticated digitalisation and community-based HMO (Health Maintenance Organisation) system is helping it lead the world in Covid vaccination. Everyone in Israel is a member of one of four HMOs. Once Israel has hold of a vaccine supply the HMOs send out text messages to eligible patients, allowing for seamless appointment making. Individuals can also book online. The HMOs track side effects by sending out a short SMS questionnaire after vaccination, and that data is aggregated and sent to the Ministry of Health (MoH). Two of the HMOs have their own research institutes, which also aggregate the data for their HMO and send out data. The HMOs hold decades-worth of medical information on each of its patients, meaning it can cross reference responses to vaccination with prior conditions. Aggregated data is then shared publicly daily.

    An Opportunity for Radical Reform

    An Opportunity for Radical Reform

    Covid-19 has laid bare the weaknesses of our data systems. Although countries like South Korea and Israel have shown best practice, very few countries have been able to take advantage of sophisticated data infrastructure to aid their response to the crisis.

    Addressing gaps in the way countries manage data will be vital to get through the current crisis and to prepare for the next. We have previously set out how countries can achieve the necessary step-change in data and situational awareness. As countries face more deaths and prolonged lockdowns, radical reform of our data systems is now more urgent than ever.

    Find out more
    institute.global