Clinical Performance of the Full Genotyping Agena MassARRAY HPV Assay Using SurePath Screening Samples within the VALGENT4 Framework

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 jmdjournal.org 63 64 65 66 67 68 69 70 71 72 73 74 Clinical Performance of the Full Genotyping 75 76 Agena MassARRAY HPV Assay Using SurePath 77 78 79 80 81 Screening Samples within the VALGENT4 Framework Helle Pedersen,* Ditte Møller Ejegod,* Wim Quint,y Lan Xu,z Marc Arbyn,z and Jesper Hansen Bonde*

Because Q7 human papillomavirus (HPV) primary cervical cancer screening is replacing cytologic analysisebased cervical screening in countries around the world, there is increased focus on triage of HPV-positive women to ensure the specificity of cervical screening. Today, cytologic analysis is the most used triage method of HPV-positive samples, yet molecular methods, such as extended genotyping, is gaining momentum as a triage method. 1e3 Different cancer-causing HPV genotypes carry distinct, differential risk of cervical cancer disease, allowing a substratification of the International Agency for Research on Canceredefined high-risk HPV group with respect to follow-up recommendations after a positive screening test result. 2 Moreover, the advantages of genotype triage are that it is embedded in the HPV testing itself and no additional reflex testing is needed to obtain the complete diagnostic outcome, which contrasts with triage strategies that involve cytologic analysis or immune cytochemistry staining. Finally, because women undergo multiple screening rounds 1 2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  60  61  62  63 over time, genotyping will allow clinicians to assess HPV persistence as a risk-defining element in the evaluation of HPV-positive women. HPV tests can be categorized into 4 assay design types 4 : i) consensus assays, which report only positive or negative outcomes that measure the presence of the 13 or 14 oncogenic HPV genotypes (HPV16, 18,31,33,35,39,45,51,52,56,58, 59, and 68 with or without HPV66); ii) consensus assays with limited (partial) genotyping reporting for HPV16 and HPV18; iii) HPV assays with extended genotyping, typically HPV16 and HPV18 combined with more but not individual reporting of all the oncogenic genotypes; and iv) full genotyping assays with individual reporting of the 14 oncogenic HPV genotypes. The number of commercially available HPV assays has increased notably in the last years, with >200 assays now on the market. 5 At the same time, HPV assay designs for cervical cancer screening has evolved from category 1 designs described above to entail increasingly more individual genotype reporting. However, only a few of these assays have been validated according to the international guidelines. 6,7 Besides cervical cancer screening, HPV assays are also used in a variety of related clinical modalities. These modalities include test-of-cure (ToC) after conization, 8 epidemiologic monitoring of HPV vaccination effects, and HPV test on a number of samples, such as vaginal self-samples and urine, 9e11 and specimen types from various medical specialties, including dermatology, venerology, head-andneck, and infectious medicine. Common to the noncervical cancer screening applications of the HPV test is the need for agile HPV assays that can run on a variety of sample types.
The Validation of HPV Genotyping Tests (VALGENT) framework is an international cooperation that was designed for clinical validation and comparison of HPV assays with genotyping capabilities for use in primary HPV screening. 4 The VALGENT panels by design constitute a cohort of consecutive cervical screening samples (screening population) and disease-enriched samples with abnormal cytologic test results (enriched population). 4 Until now, four VAL-GENT panels have been collected from well-established laboratories around Europe with screening samples from the Belgian (VALGENT1), 12e14 Scottish (VALGENT2), 15e19 Slovenian (VALGENT3), 20e27 and Danish (VALGENT4) 28e33 cervical cancer screening programs.
The full genotyping MassARRAY HPV (MA-HPV) assay from Agena Bioscience (Hamburg, Germany) was evaluated using a general primer (GP) 5þ/6þ PCR enzyme immunoassay (GP5þ/6þ EIA) as comparator, supported by the GP5þ/6þ PCR with Luminex (GP-LMNX) genotyping for individual genotypes. The assay was validated with 1294 SurePath screening samples from women 30 years of age who participated in the Danish cervical cancer screening program, which constituted the fourth VALGENT panel (VALGENT4). 28 The MA-HPV is a newly designed HPV assay based on matrix-assisted laser desorption-ionization time-of-flight mass spectrometry (MALDI-TOF MS), which has not previously been validated for HPV cervical cancer screening.

Material and Methods
Sample collection and data retrieval for VALGENT4 were approved by the Danish Data Inspection Agency. A European Union General Data Protection Regulationecompliant data handler agreement was established between Copenhagen University Hospital, Copenhagen, Denmark (the principal site), and Sciensano, Brussels, Belgium, for the data analysis. All collected samples were cross-referenced and found eligible via the Danish register (Vaevsanvendelsesregistret), relating to the collection, storage, and use of human biological material in health research projects.

Sample Collection and Histologic Follow-up
The sample collection procedure has been described in detail previously. 28 In short, the VALGENT4 panel consists of SurePath screening samples collected from women participating in the Danish cervical cancer screening program at the Department of Pathology, Hvidovre Hospital, Denmark (parent laboratory). The VALGENT panel is standardized 4 and comprises two study populations: 997 consecutive routine screening samples (screening population), comprising 946 samples negative for intraepithelial lesions or malignancy (NILM), 6 atypical squamous cells of undetermined significance, 21 low-grade squamous intraepithelial lesions, and 24 high-grade squamous intraepithelial lesions, atypical glandular cells (AGCs), atypical squamous cells (cannot exclude high-grade squamous intraepithelial lesions), or adenocarcinoma in situ (mean age, 42.4 years; range, 30 to 59 years) ( Table 1 ½T1 ½T1 ). The disease-enriched component included 297 cytologically abnormal samples (100 atypical squamous cells of undetermined significance, 100 low-grade squamous intraepithelial lesions, and 97 high-grade squamous intraepithelial lesions,) (enriched population), with a mean age of 40.4 years (range, 30 to 59 years) ( Table 1). All cytologic and histologic procedures were performed at the parent laboratory as described previously. 28 Subsequent histologic screening history on women included in the study was retrieved from the Danish PatoBank by linkage to the central personized registrar. Histologic follow-up was assessed 32 months (range, 32 to 35 months) after baseline testing and revealed 122 women with cervical intraepithelial neoplasia 2 confirmed at histologic follow-up, with most derived from the enriched population (N Z 109). A total of 897 women had two consecutive cytologic NILM samples at baseline and 12 to 24 months earlier, and these samples were used for the specificity calculations.

DNA Extraction
As previously described, 28 DNA extraction in VALGENT4 was performed at the parent laboratory using the MagNA Pure96 system from Roche Diagnostics (MagNA Pure 96 DNA and Viral NA Small Volume Kit). The mean number of days from sample reception date to DNA extraction was 27 days (range, 11 to 71 days).

MassARRAY HPV Assay Testing
The MA-HPV assay (Agena Bioscience) is a newly designed full genotyping assay based on PCR and Mas-sARRAY using MALDI-TOF MS) technology. The HPV panel allows for individual detection of 19 HPV genotypes (6,11,16,18,31,33,35,39,45, 51, 52, 53, 56, 58, 59, 66, 67, 68, and 73). In this analysis, the focus was on the 14 oncogenic HPV genotypes. The assay is performed during a 2-day period with 94 samples per run. On day 1, an initial multiple amplification PCR is set up with 2 mL of biobanked DNA, followed by a shrimp alkaline phosphatase reaction (which removes the excess nucleotides). After the shrimp alkaline phosphatase reaction, an iPLEX pro single base extension PCR (Agena Bioscience) is set up. A mix of oligonucleotide extension primers designed to anneal to the amplified DNA fragments are added together with extension enzyme and mass-modified dideoxynucleotide terminators. On day 2, the extension products are desalted with clean resin before being loaded into the MassARRAY Dx Nanodispenser RS1000 (Agena Bioscience), transferring the analyte to a spectroCHIP, which is then analyzed on the MassARRAY Dx analyzer in concordance with the manufacturer's specifications. The MA-HPV assay has an internal glyceraldehyde-3-phosphate dehydrogenase control for sample sufficiency and assay performance. One sample was found twice to be invalid for MA-HPV and was excluded from the analysis, leaving 1294 samples for the final analysis. The mean number of days from DNA extraction to MA-HPV testing was 75 days (range, 2 to 162 days).

Comparator Assay Testing
An DNA aliquot was shipped to DDL Diagnostics Laboratory (Rijswijk, the Netherlands), where all GP PCR testing was performed. The clinical validated high-risk GP-EIA for pooled detection of 14 HPV oncogenic types (16,18,31,33,35,39,45, 51, 52, 56, 58, 59, 66, and 68) was used as a comparator for the clinical validation of the MA-HPV assay. For genotyping comparison analysis, a LMNX-based readout for individual genotyping of the 14 oncogenic HPV types was used. 18 GP-LMNX testing on biobanked DNA aliquots was completed 685 days after sample reception. The GP-EIA was subsequently performed and completed on the GP5þ/6þ amplicons 1008 days after samples reception.

Statistical Analysis
A sample was considered MA-HPV positive if >1 of the 14 oncogenic types was detected, and a sample was considered MA-HPV negative for all other outcomes, including detection of low-risk HPV genotypes; the same applied for GP-LMNX. The level of genotype agreement between MA-HPV and GP-LMNX was determined using k statistics. The following definitions were used: poor, 0.00 to 0.20; fair, 0.21 to 0.40; moderate, 0.41 to 0.60; good, 0.61 to 0.80; and excellent, 0.81 to 1.00: excellent. 34 The accuracy of the MA-HPV assay for detection of CIN 2 and CIN 3 was assessed and compared with the GP-EIA. Noninferiority of the MA-HPV assay compared with the GP-EIA was assessed according to the international validation guidelines, using the 0.90 and 0.98 benchmarks for relative sensitivity and specificity, respectively. 35,36 Relative accuracy measures (MA-HPV vs GP5þ/6þ PCR) with McNemar 95% CIs were computed.
Testing results for MA-HPV, GP-EIA, and GP-LMNX were sent to the Unit of Cancer Epidemiology, Sciensano, for statistical analysis, which was performed using STATA software version 14 (StataCorp, College Station, TX).

HPV Genotyping Concordance between MA-HPV and GP-LMNX
The overall HPV concordance in the entire VALGENT4 panel was 88.1% (k Z 0.74) (Supplemental Table S1). Differentiating the screening and enriched cohorts, the overall HPV agreement was slightly higher in women with disease (Supplemental Table S1). The agreement for the individual 14 HPV genotypes for the entire VALGENT panel was fair for HPV68 and HPV59 (k Z 0.33 and 0.38), moderate for HPV51 (k Z 0.58) and HPV66 (k Z 0.54), good for eight genotypes (k Z 0.68 to 0.80), and excellent for HPV16 (k Z 0.89) and HPV35 (k Z 0.81). When limiting the analysis to the screening population, the individual genotype concordance ranged from fair (k Z 0.21) to excellent (k Z 0.82); for the enriched population, the concordance was better for almost all genotypes, with fair (k Z 0.25) to excellent (k Z 0.92) agreement.

Discussion
The MA-HPV is a MALDI-TOF MS HPV assay with full genotyping. To the best of our knowledge, no study has previously been published on the MA-HPV using cervical cancer screening samples. In this study, the clinical validation of the MA-HPV assay is conducted within the VALGENT framework 4,28 on SurePath-collected screening samples from women participating in the Danish organized cervical cancer screening program. Noninferiority for sensitivity of the MA-HPV compared with the GP5þ/6þ comparator test was found for both CIN 2 (P Z 0.0001 for noninferiority) and CIN 3 (P Z 0.0009 for noninferiority) but not for specificity (P > 0.99 for noninferiority) ( Table 4). The relative sensitivity was 1.02 (95% CI, 0.98 to 1.05) for CIN 2 and 1.01 (95% CI, 0.99 to 1.04) for CIN 3. The relative specificity was 0.89 (95% CI, 0.86 to 0.91) for NILM samples at baseline and 12 to 24 months earlier.
This finding is not surprising given that the MALDI-TOF assay runs without defined clinical cutoffs and that this technology platform allows for very sensitive analysis (ie, in the substantial detection of HPV59). However, with respect to cervical screening samples, the high sensitivity translates into substantially lower specificity.
The overall concordance between MA-HPV and GP-LMNX was good (k Z 0.74) for all 14 oncogenic HPV genotypes in the VALGENT4 panel (Supplemental Table  S1). When the analysis between the screening and enriched population was differentiated as individual sample sets, the overall concordance was moderate (k Z 0.60) for the screening and good (k Z 0.63) for the enriched population set. When the individual specific genotype concordance is considered, a more diverse picture emerges, with k values ranging from fair (k Z 0.21) to excellent (k Z 0.82) for the screening samples and from fair (k Z 0.25) to excellent (k Z 0.92) for the enriched population. The concordance between MA-HPV and GP-LMNX was higher *P < 0.05 for noninferiority means that the accuracy of MA-HPV is not significantly lower than that of GP-EIA, using the benchmarks of 0.90 for relative sensitivity and 0.98 for relative specificity. Noninferiority is significant for sensitivity but not for specificity.  621  622  623  624  625  626  627  628  629  630  631  632  633  634  635  636  637  638  639  640  641  642  643  644  645  646  647  648  649  650  651  652  653  654  655  656  657  658  659  660  661  662  663  664  665  666  667  668  669  670  671  672  673  674  675  676  677  678  679  680  681  682  683   684  685  686  687  688  689  690  691  692  693  694  695  696  697  698  699  700  701  702  703  704  705  706  707  708  709  710  711  712  713  714  715  716  717  718  719  720  721  722  723  724  725  726  727  728  729  730  731  732  733  734  735  736  737  738  739  740  741  742  743  744 for almost all genotypes in the enriched population (Supplemental Table S1). The higher concordance between HPV assays in women with disease has previously been observed in other studies. 29,30,37 The observation is that HPV16 and HPV35 has the best concordance between the MA-HPV test and the comparator assay, whereas HPV68 has the poorest concordance. In addition, HPV68 is close to inconsequential in cervical cancer screening with respect to cancer risk despite being included on the International Agency for Research on Cancer high-risk list. 2,38 The MA-HPV is a full genotyping HPV assay that individually reports on 19 HPV genotypes, and only a handful of full genotyping assays have been validated for clinical accuracy according to the international guidelines. 20,23,24,26,39,40 Most full HPV genotyping assays require cutoff optimization to fulfill validation criterions for use in cervical screening. 23,25,26,32,33,40 This requirement is typically a consequence of the assay design in which the manufacturers aim for high sensitivity of detection in a variety of sample types (ie, formalin-fixed, paraffinembedded tissue samples, swab samples, and samples collected in liquid-based cytologic media). However, this focus on high sensitivity collides with the requirements of cervical cancer screening in which sensitivity and specificity must be balanced to reduce overdiagnosis.
The MA-HPV assay cannot be easily cut off optimized for use with cervical screening samples. This is due to the nature of MALDI-TOFF technology which basically measures time-of-flight through a vacuum tube by laser evaporated DNA strands of defined length. Yet, the MA-HPV test system provides an agile platform with a reasonable throughput for advanced HPV genotype detection constituting a clear competitor to NGS as the MA-HPV does not require bioinformatics to obtain the final test results. Looking across the field of cervical screening, the ability to provide highly sensitive and detailed HPV genotype results could be utilized in a subset of screening derived samples. Specifically, most organized screening program require an HPV analysis after treatment (ToC). Often these samples are analyzed using the same HPV assay as employed in primary HPV screening. However, the underlying question is different. In ToC, the aim is to detect residual HPV infection in the cervix post conization and, if possible, determine whether continued HPV positivity constitute same-genotype persistence or a genotype switch. 2,41 Whereas a high-risk HPV test result is sufficient to predict recurrence with high sensitivity, 8,41e43 its specificity is low and might be increased by identifying the same types before and after conization 8,44 ?
In particular, full genotyping assays, such as the MA-HPV assay, are operated independently of sample type and sample collection media given that they analyze extracted DNA. In the case of ToC testing, HPV testing with any biopsy material and the ToC sample together compared with the original screening sample Q8 could help assess a woman's risk of recurring disease. The ability to run on extracted DNA independent of sample types provides versatility because the DNA input material can be obtained from many different platforms and extraction kits. Moreover, the MA-HPV assay requires a comparatively small amount of DNA input material, making the assay useful for HPV analysis on samples with scarce material (ie, small biopsy specimens, whether fresh or formalin fixed).
The strength of the VALGENT4 study is that the samples were freshly collected SurePath samples from the Danish women in the primary screening target age range of 30 to 59 years who participated in organized cervical cancer screening with complete registration of follow-up. Another strength was that extracted DNA used for the MA-HPV, GP-EIA, and GP-LMNX testing was extracted at the parent laboratory and aliquoted from the same stock, thereby avoiding any bias introduced by different preanalytical DNA extraction methods.
A weakness of the study, also discussed in the published protocol, 28 was the sample preparation protocol, which deviates from routine practice. However, because the MA-HPV and comparator assays use preanalytical extracted DNA rather than original sample medium, the effect of this is expected to be minor. This expectation is also supported by the low number of invalid samples by MA-HPV, GP-EIA, and GP-LMNX in this study and other assays validated in the VALGENT4 study. 29e33 In conclusion, the MA-HPV assay is a clinically sensitive assay with a lower clinical specificity than the comparator assay. The lower specificity could be attributable to the detection of HPV infections at low viral loads, and a cutoff optimization could theoretically improve the low specificity in a screening setting. However, the assay in its current form seems more suited to play a role in evaluation of viral infection outcomes where specificity is of less importance but where sensitivity of detection is paramount. These applications include HPV genotype monitoring in epidemiologic studies of HPV vaccination outcomes, clinical TOC evaluation after conization, and clinical noncervical screening diagnostic samples for HPV testing.  869  870  871  872  873  874  875  876  877  878  879  880  881  882  883  884  885  886  887  888  889  890  891  892  893  894  895  896  897  898  899  900  901  902  903  904  905  906  907  908  909  910  911  912  913  914  915  916  917  918  919  920  921  922  923  924  925  926  927  928  929  930  931