Questions about COVID-19 test accuracy raised across the testing spectrum

For Sarah Bowen, it all started with a sore throat. Not the kind of searing pain she’d feel with strep, she said, but a throat irritation that just didn’t feel right.

“By the end of the day, it just got a little worse and I didn’t feel great. I felt like I might be coming down with something. And the next day, things got worse,” Bowen, 31, of Portland, Oregon, said.

Full coverage of the coronavirus outbreak

Bowen works at a doctor’s office, where she was immediately able to get tested for COVID-19, on May 8. It came back negative, and her doctor said the symptoms were most likely allergies or another virus.

But from there, things snowballed. Bowen developed headaches, a stuffy nose, hot flash symptoms and constant headaches. By day six, she felt like she was hit by a truck. She had extreme fatigue and a burning sensation in her chest.

“I started getting shortness of breath if I went upstairs to get water or something,” Bowen said. “It got worse when I moved around.”

Two days later, she took another test for COVID. Again, it came back negative.

But despite her symptoms, her doctor didn’t believe she had the virus, because there weren’t many cases in the Portland suburb where she lives. Frustrated, Bowen continued to isolate alone in the downstairs of her home. She didn’t want to take any chances. 

“It’s one thing to get sick and know it’s a cold or the flu. But to get sick during a pandemic and to be kind of dismissed, makes you feel crazy,” she said.

Bowen’s diagnosis remains unclear, but her experience raises questions about the accuracy of diagnostic tests for the disease. Indeed, as more and more people have access to testing, new data show that false negatives on COVID-19 tests may be more common than first realized.

And as the U.S. starts to reopen, accurate testing is one of the most important tools in states' arsenals to track — and stop — the spread of the coronavirus.

Accuracy issues

Since the pandemic started spreading across the United States in March, nearly 70 tests have received emergency use authorization from the Food and Drug Administration. Many of these tests were developed at a breakneck pace in an effort to get tests out to the American people.

But while no test is perfect, experts told NBC News that these particular tests — used to diagnose COVID-19 — may be missing up to 20 percent of positive cases.

One key reason behind these so-called false negatives may be how the testing samples are collected.

“The false negatives are mainly due to specimen acquisition, not the testing per se,” said Dr. Alan Wells, medical director for the University of Pittsburgh Medical Center clinical laboratories and a professor of pathology at the University of Pittsburgh.

Most tests use a method called polymerase chain reaction or PCR. It detects coronavirus genetic material that’s present when the virus is active. Clinicians typically collect a sample for testing from the back of a person’s throat — where the virus is presumed to be — with a long nasopharyngeal swab.

But scientists say that collection method is ripe for error.

“You’re sampling blindly. You’re hoping you get the right spot. Then as the disease progresses, the virus might migrate down into your lungs,” Wells said, adding that once it’s in the lungs, that nasopharyngeal swab may not pick up any virus if it’s already been cleared from the throat.

“You have to be at the right place at the right time,” he said.

Another type of diagnostic test forgoes the uncomfortable swab altogether, and instead uses saliva collected in a test tube. Once the sample arrives in the lab, it’s tested the same way, with PCR.

But Wells said those tests could fare even worse.

“The reason for pharyngeal swabs is the virus preferentially infects and replicates starting way back in the inner cavities of the nose and not out in front,” where it may come into contact with saliva, he said, adding that saliva tests could end up missing up to 50 percent of asymptomatic positive cases.

Making things even more complicated, a May 13 study in Annals of Internal Medicine, from researchers at the Johns Hopkins Bloomberg School of Public Health in Baltimore, found that test timing is also essential to getting an accurate result.

Lead study author Dr. Lauren Kucirka, a medical resident at Johns Hopkins Medicine, said testing too early after exposure to the virus substantially raises the risk of a false negative.

“If you have someone who has been exposed and they’ve started to develop symptoms, it probably makes sense to wait a few days before testing,” Kucirka told NBC News.

Her study found that three days after the onset of symptoms is when the test is most likely valid.

But besides issues with how and when test samples are collected, questions are also being raised about the quality of the diagnostic tests themselves.

The biggest problem with that is you create a false sense of security.

In other words, even if samples are collected perfectly, at the ideal time, the tests could turn up incorrect results. A commentary published in April in Mayo Clinic Proceedings criticized the reliance on PCR tests, saying that even when tests are 90 percent accurate, that still leaves a substantial number of false test results.

The article’s co-author, Dr. Priya Sampathkumar, an infectious disease specialist at the Mayo Clinic, used California as an example in a statement: If the entire population of 40 million people were tested, there would be 2 million false negative results. Even if only 1 percent of the population was tested, there would be 20,000 false negatives.

“The biggest problem with that is you create a false sense of security,” Wells said.

Not just PCR problems

Another type of COVID-19 diagnostic test, Abbott Labs’ popular ID NOW point-of-care test, has also come under fire in recent weeks, after the FDA issued an alert that it may not always be accurate.

The test, which uses a method different from PCR, called isothermal nucleic acid amplification, can deliver results in five to 13 minutes. It’s used by doctors across the country and touted by the White House as what’s used to test President Donald Trump and other staffers.

One small study by NYU Langone Health found that the test returned false negatives for nearly 50 percent of certain samples that a rival test had found to be positive. The study has not yet been peer-reviewed.

In response, Abbott last week released interim data on several of its own studiesfinding that accuracy was significantly better, in some cases nearly 100 percent, especially when performed in patients who were tested early after their onset of symptoms.

But anecdotal reports have also found issues with accuracy, leading some of the nation’s largest medical centers to stop or never even start using it.

NBC News spoke with 10 medical centers and hospitals across the country; seven said they weren’t using the Abbott test. 

All seven cited issues with accuracy, including Jackson Memorial Hospital System in Miami, which said in a statement that they “identified some issues with the accuracy, which is to be expected when the medical science is so new and evolving so quickly around this virus. The best fit for Jackson was to transition to other testing platforms that have high-quality accuracy rates and quick turnaround times for results.”

A Vanderbilt University Medical Center spokesman told NBC News that “No patient at Vanderbilt University Medical Center has been tested via the Abbott ID NOW rapid test. Here, there were concerns about the sensitivity of that test.”

Some hospitals continuing to use the Abbott test, such as Sutter Health Hospitals in California, said they often will confirm any negative results with another PCR test if there is clinical suspicion of COVID-19.

Abbott told NBC News in a statement that to date, the company has delivered more than 2 million tests to all 50 states.

"Our customers are telling us that they’re seeing positivity rates from ID NOW testing at or above local community infection rates, which means that ID NOW is detecting the virus at the same level as lab-based testing," the statement said in part. "If there were any systemic problem with ID NOW producing false negatives, that wouldn’t be the case."

The bigger issue may be that test manufacturers just haven’t caught up to science. It’s not just COVID-19 tests that have issues with accuracy. In fact, diagnostic tests for all sorts of common diseases are not even close to perfect.

Take rapid strep throat tests, for instance. According to a Cochrane Review, those tests have a sensitivity of just 86 percent. The Centers for Disease Control and Prevention says rapid flu tests are even worse, with a sensitivity ranging from 50 to 70 percent.

Rapid strep and rapid flu tests look for antigens — proteins made by the infectious pathogen — rather than genetic material. The first antigen test for COVID-19 received an emergency use authorization from the FDA earlier this month, but questions have already been raised about its accuracy.

Taken together, it’s why Dr. Ania Wajnberg, associate director of medicine at the Icahn School of Medicine at Mount Sinai, said that diagnostic tests need to be put together with clinical suspicion.

“We still have a lot to learn, but testing itself is hugely important,” Wajnberg said. “If it’s not perfect, it doesn’t mean it’s not useful.”