Serological tests are critical in the context of infectious disease outbreaks because they have the potential to identify the true prevalence of infection in a population, allowing measures such as the infection fatality rate to be accurately calculated. This property is based on the assumption, however, that serum antibodies remain detectable for long periods after an infectious agent is eliminated from the host. While well-grounded on our collective experience with other viral infections,1-3 the validity of this assumption in the case of SARS-CoV-2 is less clear. A number of studies, including one from our group, have shown declines over time in SARS-CoV2 antibody titers or percent positivity using various assays, either at the individual or population levels,4-20 although other studies reported stable antibody levels.21-24 Comparison of these results is confounded by the use of different antibody assays, including rapid lateral flow tests, ELISAs, and commercial chemiluminescence tests, all of which have different and often poorly-defined cut-off points for determining seropositivity.

The first COVID-19 cases in Brazil were reported in February 2020. In the State of Rio Grande do Sul, the first case was confirmed on March 10, and a series of regular population-based antibody surveys was started on April 11 in nine cities.25 In early May, the study was expanded to include 133 large cities in Brazil's 27 federation units, covering an area of approximately 4200 by 4000 km; so far four data collection rounds were completed, with over 25,000 individuals having been tested in each phase.26 At the time the studies were launched, the only antibody test available in large numbers in Brazil was a rapid lateral-flow test (Wondfo SARS-CoV-2 Antibody Test, Wondfo Biotech Co., Guangzhou, China), which had been purchased by a large company and donated to the Ministry of Health. The first three rounds of the national study, carried out in May and June, showed a steady increase in uncorrected prevalence from 1.9% to 3.8% (95% CI 3.5–4.1) over a five-week period. For the fourth round, carried out in August 2020, the research team no longer had access to the original batch of tests, and it was necessary to purchase 50,000 units from the sole Brazilian company that imported the Wondfo test. Results from the fourth phase showed a surprisingly low prevalence of antibodies, of only 1.4% (95% CI 1.2–1.6), at a time when the numbers of reported cases and deaths had been steadily increasing in most of the country.

Although at the time of the fourth round of the national survey, there was already evidence from the literature that antibody titers in large-scale studies had been falling over time according to some studies,4-10 the marked reduction observed in our survey was unexpected. As a consequence, we launched a study to compare how the sensitivity of the two batches of the Wondfo test vary with time since PCR-confirmed diagnosis of COVID-19. We also took the opportunity to assess the validity of a new low-cost ELISA test developed in Brazil that relies on dried blood spots collected on filter paper that may represent a practical and affordable alternative for large-scale epidemiological studies.

Following ethical approval by the Brazilian National Ethics Committee (process number 30415520.2.0000.5313), local health authorities, hospitals and laboratories in the city of Pelotas, Brazil, were requested to allow access to lists of patients with positive PCR tests for SARS-CoV-2 from March to October 2020. Patients were contacted by phone, and starting from these contacts, snowball sampling was used to obtain a larger number of subjects who had tested positive. All participants signed informed consent forms before providing samples.

Data collection was conducted in October and November 2020. Subjects were visited at home, at least 14 days after the original PCR result. A trained team of health visitors, wearing personal protection equipment, obtained blood samples by fingerpick draw. Three tests were performed on each subject: an in-house direct ELISA to detect IgG to the SARS-CoV-2 spike (S) protein from dried blood spot samples (S-UFRJ ELISA)27 and two different batches of the Wondfo SARS-CoV-2 Antibody Test, which detects antibodies to the S-protein's receptor binding domain (RBD).

First, three drops of blood were collected on three replicate filter paper pads attached to a plastic strip and allowed to dry. Samples were sent to the Biotechnology Laboratory at the Federal University of Pelotas for ELISA testing. Eluates were assayed using the S-UFRJ ELISA protocol as described previously.27 The S-UFRJ ELISA displayed 98.4% specificity and sensitivity above 90%.27 Briefly, high-binding ELISA plates (Corning) were coated with 50 μL of SARS-CoV-2 S protein at 4 μg/mL suspended in PBS and incubated overnight. The coating solution was removed and 250 μL of PBS 1% BSA (blocking solution) was added to the plate and incubated at room temperature for 1–2 h. Meanwhile, single filter paper pads cut from the plastic strips were used to prepare eluates by incubating for one hour at room temperature in 200 μl of PBS 1% BSA. The blocking solution was then removed and 50 μL of eluate was added to the plate and incubated at room temperature for two hours. The plate was then washed five times with 250 μL of PBS. Next, 50 μL of 1:10,000 goat anti-human IgG (Fc)-horseradish peroxidase antibody (Rhea Biotech) was added to the plate and incubated for 1.5 h at room temperature. The plate was then washed five times with 250 μL of PBS. Finally, TMB (3,3 ', 5,5; -tetramethylbenzidine) (Thermo Fischer) horseradish peroxidase substrate was added, and the reaction allowed to develop for 15 min; the reaction was stopped with 50 μL of 1 N HCl, and optical density (OD) was read at 450 nm with 655 nm background compensation in a microplate reader (Biochrom EZ read 400). Relative levels of antibodies are expressed as OD ratio of values of individual samples to the mean plus three standard deviations of the OD average of the negative controls in the same ELISA plate. An OD ratio below 0.9 indicates a negative result, an OD ratio above 1.1 indicates a positive result, and an OD ratio between 0.9 and 1.1 is considered undetermined, which is compatible with the range used in most commercial diagnostic ELISA tests. Because samples were assayed at a single dilution, data were plotted as a dichotomous rather than a quantitative variable, although the test also allows quantification of antibody titers using multiple dilutions.

After collection of filter paper samples, each subject provided additional drops of blood for two different batches of the Wondfo SARS-CoV-2 Antibody Test (Wondfo Biotech Co., Guangzhou, China). Batch A was part of 500,000 kits imported in March 2020 directly from China by the Brazilian Ministry of Health, with funding from Vale do Rio Doce, a private mining company (batch numbers W19500433, W19500450, W19500498 and W19500460). Batch B (number W195004116) was part of a lot of 50,000 kits purchased by the Federal University of Pelotas directly from the Wondfo representatives in Brazil. The use-by dates of the two batches were October 2020–April 2021, and April 2021, respectively.

This is a rapid point-of-care lateral-flow test that detects IgG and/or IgM isotypes specific to the RBD portion of the SARS-CoV-2 S protein. Test results were read after 15 min by the field worker and the kits were photographed for a second independent reading by a supervisor (MFS). According to the manufacturer, the test's sensitivity and specificity are 86.4% (95% CI 82.4–89.6) and 99.6% (95% CI 97.6–99.9), respectively. By pooling the results from four available validation studies, one of which was carried out by our team before the first survey round,28 we estimated a sensitivity level of 84.8% (95% CI 81.4–87.8). Specificity was estimated at 99.95% based on an early survey we carried out in a population-based sample of 4188 subjects at an early stage of the epidemic in Rio Grande do Sul state25; we considered that the two individuals with positive results were false positives.

All subjects were classified according to the calendar month when the positive PCR tests had been carried out. Results of the three different tests were tabulated according to the month of the PCR.

We tested 133 subjects with positive PCR results from March to October 2020. There were no refusals. Because there were few cases in the city during the first half of the year, the five cases diagnosed from March to May were pooled with the 26 cases from June. The numbers of cases by month are shown in Table 1. Of the 133 subjects, 77 were women and the mean age was 41.7 years and the standard deviation 15.9 years (range 3 to 88 years).

Of the 133 ELISA tests, 123 were positive, seven inconclusive and three negative. For further analyses, inconclusive tests were recorded as negative. For Wondfo A tests, 84 were positive, and 45 for Wondfo B.

Sensitivity results, using PCR as the gold standard, are shown in Table 1 and graphically in Fig. 1. Overall sensitivity was 92.5% for ELISA, 63.2% for Wondfo batch A, and 33.8% for Wondfo batch B. Whereas sensitivity of the ELISA test remained stable at around 90%, with no evidence of a decline, both Wondfo tests showed marked reductions in sensitivity over time. The confidence intervals for the three tests (Table 1) are rather wide due to the study's sample size. Nevertheless, none of the 95% CI's for the Wondfo B test overlap with those from the ELISA test.

Fig. 1.

Sensitivity of the three tests for COVID-19 according to date of the original PCR result.

(0.24MB).

Compared to the ELISA test, sensitivity of Wondfo A was 66.7% (82/123) and of Wondfo B 36.6% (45/123). Using Wondfo A as the gold standard, the sensitivity of Wondfo B was equal to 47.6% (40/84), ranging from 65.0% in October to 46.2% in July. Chi-squared tests for linear time trends in proportions showed P levels of 0.802 for ELISA, 0.001 for Wondfo A and 0.011 for Wondfo B.

Table 2 shows that there was little variation in the sensitivity of the ELISA and Wondfo B tests by age and sex of the subject, but sensitivity of the Wondfo A test appeared to be lower in women (P=0.062) and in individuals aged 40–49 years (P=0.072).

We have used the Wondfo test in repeated population-based surveys carried since April 2020,25,26 when it was the only test available at large scale in Brazil. Its advantages included producing results within 15 min, thus allowing field work to be completed within four days in 133 cities with a sample size of over 30,000 subjects. Our choice was supported by early validation studies – including our own28 - showing a level of sensitivity around 80%, with very high specificity. A comparison of 12 rapid tests carried out at the University of California San Francisco showed that the Wondfo test used in our study was among the ones with the best performance, with sensitivity over 80% and specificity over 99%.29 In the present analyses, we also found sensitivity levels of around 80% in recently diagnosed COVID-19 cases (Fig. 1), using the same batch of the Wondfo test we used in April during the original validation.28

At the early stage in the pandemic, it was widely assumed that seropositive individuals would remain so for many months,30 as had been observed during the original SARS-1 epidemic,31,32 most4-20 - but not all21-24 – studies started to report rapid drops in antibody levels. Within the first three rounds of our nationwide survey (May-June), we had already noticed substantial drops over time in high-prevalence Amazon cities which were the most affected early in the pandemic.26 An even more dramatic drop was noted in August, when 1.4% of individuals tested positive compared to 3.8% in June. This observation prompted the present analyses.

The validation analyses confirmed our suspicion of a systematic difference between two sets of batches. Compared to the gold standard PCR results, sensitivity of the first and second batches were equal to 63.2% and 33.8%, respectively, with both batches showing rapid declines over time. Earlier studies comparing the performance of different brands of rapid tests had shown substantial variability,29 but to our knowledge this is the first evidence of marked differences among different batches of the same test brand, not only for recently diagnosed patients but also over time. Batch-related issues may have contributed to the discrepancies between studies measuring the sensitivity of the Wondfo test in different settings. Further batch-by-batch validation studies of similar lateral-flow, point-of-care tests are warranted.

In light of the recent development of the S-UFRJ ELISA test in Brazil, we took the opportunity to also assess its sensitivity in the same study. Unlike the rapid tests, ELISA showed high sensitivity of 92.5% overall, and no evidence of a decline over the five months of the study. The ELISA test has very low cost (about US$1.00 per test including lab costs) and is performed on dried blood spots. These characteristics suggest that this test, and possibly other similar ELISA tests, may be advantageous over point-of-care tests in large epidemiological studies. For comparison purposes, the price paid for the each Wondfo B test in Brazil was around US$5.00.

Although the reasons for the discrepancy between the Wondfo and S-UFRJ test are unclear, two factors may play a role. First, the Wonfdo test detects both IgM and IgG antibodies, the first of which decay more rapidly with time post-infection.10,33,34 If positivity in the Wondfo test is driven primarily by high-avidity IgM interactions, loss of positivity with time is to be expected. Second, a recent report34 indicates that antibodies specific to the RBD, detected by the Wondfo test, decay more rapidly than those binding to other regions of the S protein, the antigen of S-UFRJ. Thus, measuring anti-RBD antibodies, while advantageous when attempting to assess neutralization potential, may not be ideal for assessments of antibody prevalence in a population.

Our findings cast serious doubts about the use of this brand of rapid antibody tests for epidemiological studies. We are now using the curves of test positivity by time since the initial PCR to correct the observed antibody prevalence in each phase of Rio Grande do Sul and Brazil studies. The correction factor considers declining positivity over time, as well as the epidemic curve (based on reported deaths and cases) in each city.

Our findings raise the limitations of commercially available point-of-care tests for use in epidemiological studies, while suggesting that paper-filter ELISA tests may represent a valid alternative. Without the need for venopuncture and at low cost, the S-UFRJ test showed high sensitivity that is preserved over several months.

The study was funded by the “Todos Pela Saúde” initiative, Instituto Serrapilheira, Brazilian Ministry of Health, Brazilian Collective Health Association (ABRASCO) and the JBS S.A. initiative ‘Fazer o Bem Faz Bem’.

Role of the funding sources. The funding sources did not have any role in the analyses or preparation of the manuscript.