Real-life head-to-head comparison of performance of two high-throughput automated assays for detection of SARS-CoV-2 RNA in nasopharyngeal swabs: the Alinity m SARS-CoV-2 and cobas 6800 SARS-CoV-2 assays


 Abbott’s Alinity m is a recently launched fully integrated, automated molecular analyzer allowing continuous loading of samples and sample-to-result molecular detection of several microorganisms. Manufacturer-independent clinical evaluation of Alinity m SARS-CoV-2 (Alinity) against cobas 6800 SARS-CoV-2 (cobas) as a standard comparator was performed on 2,157 consecutive nasopharyngeal swabs. Valid initial results of Alinity and cobas were obtained from 2,129/2,157 (98.7%) and 2,157/2,157 (100%) samples, respectively. Overall percent agreement of 98.3% (2,092/2,129; 95%CI:97.6–98.7%), positive percent agreement of 100.0% (961/961; 95%CI:99.6–100.0%), negative percent agreement of 96.8% (1,131/1,168; 95%CI:95.7–97.7%), and a high kappa value of 0.965 (95%CI:0.954–0.976) were observed on 2,129 samples with valid results for both assays. There were 37 discordant results and based on discordant analyses, including previous and/or follow-up PCR results, 22 could be considered Alinity analytically true positive with high probability. Due to the lack of additional information and inability of repeated/further testing, the status of the remaining 15 discordant samples remains unresolved. Comparative real-life throughput of both analyzers assessed while testing 564 samples in parallel across two 8-hour shifts and comparative turnaround time assessment measured while processing in parallel the first 94 routine samples received in the laboratory each working day for 5 consecutive days showed similar real-life performance of both analyzers with certain differences, which has potential importance in some laboratory settings.



INTRODUCTION
Coronavirus disease 2019 (COVID- 19) pandemic has affected over 135 million people, with over 2.9 million COVID-19-related deaths as of 10 April 2021. Highly reliable laboratory diagnostics for COVID-19 are essential for case identification, patient management, and contact tracing. Detecting severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) RNA in nasopharyngeal swabs is still considered the COVID-19 reference laboratory diagnostic standard. 1,2 Although several commercial SARS-CoV-2 RNA assays have received U.S. Food and Drug Administration (FDA) emergency-use authorization (EUA), only a few have been designed for analyzers with high sample throughput to cope with the unprecedented demand for SARS-CoV-2 RNA testing and allow significant scaling up due to the fully automated sampleto-result solution. 3 The cobas 6800 System and cobas 8800 System (Roche Molecular Systems, Branchburg, NJ, USA) are fully integrated and automated analyzers allowing sample-to-result qualitative and quantitative molecular detection of several microorganisms. The U.S. FDA recently approved a range of cobas 6800/8800 System molecular assays (https://www.fda.gov/medical-devices/vitro-diagnostics/nucleic-acid-based-tests, last accessed April 10, 2021), including assays for SARS-CoV-2, which received FDA EUA on 12 March 2020. Several studies have evaluated the performance of this highly reliable and robust SARS-CoV-2 assay, [4][5][6][7][8][9][10][11] and it has become a primary comparator in many performance evaluations of novel SARS-CoV-2 RNA assays.
Alinity m (Abbott Molecular, Des Plaines, IL, USA) is another recently launched fully integrated, automated sample-to-result molecular analyzer allowing continuous loading of samples and random-access testing. Seven molecular assays have been developed for use with Alinity m, [12][13][14][15][16][17][18][19][20][21][22][23] including an assay for SARS-CoV-2, which received U.S. FDA EUA on 11 May J o u r n a l P r e -p r o o f 5 2020. Unlike the cobas SARS-CoV-2 assay with much-published performance data, peerreviewed literature contains only limited verification and validation data for the Alinity m SARS-CoV-2 assay. 20 Here the results of the large manufacturer-independent clinical evaluation of the Alinity m SARS-CoV-2 assay against the cobas 6800 SARS-CoV-2 assay as a standard comparator are presented. The head-to-head study evaluated the clinical performance of two high-throughput automated assays on 2,157 consecutive nasopharyngeal swabs in routine diagnostic settings. In addition, comparative real-life throughput assessment of two analyzers on 564 samples across two 8-hour shifts as well as comparative turnaround time assessment on the first 94 samples received in the laboratory each working day for 5 consecutive days was performed.

MATERIALS AND METHODS
Samples used for head-to-head clinical comparison. Clinical performances of the Alinity m SARS-CoV-2 and cobas 6800 SARS-CoV-2 assays were compared on a total of 2,157 unselected nasopharyngeal swabs routinely collected during a 10-day period in mid-January 2021. All samples were obtained from patients treated in the University Clinical Center Ljubljana, the largest tertiary hospital in the country with 2,138 beds which is also the principal Slovenian COVID-19 hospital with the largest COVID-19 intensive care unit. Samples were collected in a 3 ml commercial transport medium VTM (Liofilchem, Roseto degli Abruzzi, Italy) from 2,157 individuals referred for COVID-19 testing during the study period. The median transport time of samples from the collection site to the laboratory was 1 hour 51 minutes. Upon arrival at the laboratory, swabs were vortexed for 1 minute at maximum speed and two VTM J o u r n a l P r e -p r o o f aliquots were prepared: 800 µl for Alinity m SARS-CoV-2 testing and 700 µl for cobas 6800 SARS-CoV-2 testing.
Alinity m SARS-CoV-2 testing. The Alinity m SARS-CoV-2 assay is a real-time reverse transcriptase (RT) PCR-based assay for the qualitative detection of SARS-CoV-2 RNA in nasal, nasopharyngeal, and oropharyngeal swabs. 20 By using target-specific fluorescent-labeled oligonucleotide probes, the assay allows simultaneous detection and amplification of two SARS-CoV-2-specific sequences targeting the RdRp gene and N gene reported as a combined signal in one channel and individually an internal control (IC) target sequence for evaluation of sample extraction and amplification efficiency reported separately in another channel. The assay is performed on Alinity m-a fully integrated, automated molecular analyzer that allows continuous loading of samples and performs sample preparation, RT-PCR assembly, amplification, detection, calculation, and reporting of the results. Alinity m provides two different reports: "Result" (each signal is reported either as "Not Detected" if a specimen cycle number (CN) is not generated or "CN value" if CN is less than 42) and "Interpretation" (either Negative or Positive). Prior to transferring the results into the laboratory information system (LIS), the results can be reviewed directly in the system or as a printed report.
For this study, 800 µl sample aliquots were transferred into Alinity m Aliquot Tubes, loaded on Alinity m, and tested following the manufacturer's instructions.
Cobas 6800 SARS-CoV-2 testing. The cobas SARS-CoV-2 assay is a two-target RT-PCR for qualitative detection of SARS-CoV-2 RNA in nasopharyngeal and oropharyngeal swab samples. N-glycosylase system for prevention of PCR contamination. The assay is performed on either the cobas 6800 System or cobas 8800 System instrument, which consists of the sample supply module, transfer module, processing module, and analytic module. Automated data management is performed by the manufacturer's software, which assigns test results. The results can be reviewed directly on the system screen, printed as a report, or transferred to an LIS. According to the manufacturer's instructions, a tested sample was considered SARS-CoV-2 positive if cobas showed positive results either for both the ORF1 (target 1) and E (target 2) genes or for the

RESULTS
Head-to-head clinical comparison. Out of 2,157 nasopharyngeal swabs tested, valid initial results of Alinity m SARS-CoV-2 and cobas 6800 SARS-CoV-2 were obtained from 2,129/2,157 (98.7%) and 2,157/2,157 (100%) samples, respectively. Total of 28 samples were excluded from further analysis due to initial invalid Alinity m SARS-CoV-2 results; all invalid results were due to no amplification of the target and internal control failure. Table 1  Thirty-seven discordant results were identified. Discordance between the assays was completely unidirectional (Table 1) Alinity m SARS-CoV-2 internal control is a noninfectious Armored RNA sequence unrelated to the SARS-CoV-2 sequence and is introduced into each specimen at the beginning of the sample preparation. Internal control is simultaneously amplified by PCR to demonstrate that the entire testing procedure has proceeded correctly for each sample. In this study, 28/2,157 (1.3%) samples showed initial invalid Alinity m SARS-CoV-2 results, all due to no amplification of target and internal control failure. Insufficient leftover volume allowed retesting of only 10/28 study samples with initial invalid Alinity m SARS-CoV-2 results, and after retesting a valid Alinity m SARS-CoV-2 result was obtained in 9/10 (90%) of initially failed samples. Currently, the cobas 6800 System has been installed in our laboratory for 2 years and Alinity m was J o u r n a l P r e -p r o o f installed a week before the study. Although SARS-CoV-2 RNA invalid results were also initially an issue with the cobas 6800 System 4 and were mainly caused by clots, mucus, or physical contamination detected by the instrument during sample aspiration or insufficient sample volume identified in sample tubes or processing plates, the instrument soon "stabilized," and no significant problems with invalid results on the cobas 6800 System have been recorded in 10 months while routinely testing 120,000+ samples using the cobas 6800 SARS-CoV-2. After closing the head-to-head study, we continued monitoring the rate of invalid results on Alinity m

J o u r n a l P r e -p r o o f
When selecting a SARS-CoV-2 RNA assay, virologists must consider not only sensitivity and specificity, but also sample throughput, time-to-result, test complexity, reagent and instrument availability, and cost per reportable result. Assay throughput is especially a crucial parameter for large-scale testing. Alinity m and the cobas 6800 System are both fully integrated and automated analyzers allowing sample-to-result detection of SARS-CoV-2 RNA, but this comparative throughput assessment showed potentially important differences (Fig. 1). Alinity m produced the first reportable results much sooner than the cobas 6800 System (2:35h vs. 3:40h), but the cobas 6800 System finished testing 564 samples in parallel almost 2 hours earlier than Alinity m (11:35h vs. 15:30h). Similarly, although the total hands-on time for testing 564 samples was almost equal for both instruments, the cobas 6800 System required slightly less total instrument handling time than Alinity m, but slightly more total sample handling time. In addition, during testing of 564 samples in parallel across two 8-hour shifts, the cobas 6800 System required the presence of laboratory staff during seven similar time slots whereas Alinity m required staff presence during 12 varying time slots (Fig. 1). Similar potentially important differences between the analyzers were also recorded during parallel routine processing of the first 94 samples received at the laboratory each working day for 5 consecutive days. This test on 470 samples showed that TATs for most samples were somewhat shorter for Alinity m in comparison to the cobas 6800 System. This discrepancy was mainly due to differences observed in total preanalytical time between the analyzers for most samples and because our laboratory routinely (and most economically) processed 12-sample batches for Alinity m and 94-sample batches for the cobas 6800 System. Thus in different laboratory settings the differences between the analyzers observed in this study could be less evident and less important.

J o u r n a l P r e -p r o o f
When asked to qualitatively compare their experience with both analyzers for routine detection of SARS-CoV-2 RNA, laboratory staff agreed that the main comparative advantages of Alinity m are rapid TAT for smaller batches and flexible STAT prioritization, and the main disadvantages are the need to pre-thaw and centrifuge reagents, more frequent instrument interactions and sample loading/unloading, and limited (48-hour) onboard stability of positive and negative controls. The main comparative advantages of the cobas 6800 System were higher 24-hour throughput, ready-to-use reagents not requiring thawing or mixing, and less frequent instrument interactions and sample loading/unloading, and the main disadvantage (in our laboratory setting) was needing sample centrifugation to avoid problems during sample aspiration due to clots, mucus, or physical contamination.
In conclusion, the results of the manufacturer-independent evaluation of Alinity m SARS-CoV-2 on 2,157 samples in routine diagnostic settings against cobas 6800 SARS-CoV-2 showed that Alinity m SARS-CoV-2 is a reliable assay for the qualitative detection of SARS-CoV-2 in nasopharyngeal swab samples. Assays showed excellent overall, positive, and negative percent agreements with a high kappa value. Slightly higher analytical sensitivity of Alinity m SARS-CoV-2 was clinically beneficial in a limited number of samples. Comparative real-life throughput and TAT assessments showed similar performance of both assays with performance differences, which could be potentially important in some laboratory settings.