Advertisement

Creation of an Expert Curated Variant List for Clinical Genomic Test Development and Validation

A ClinGen and GeT-RM Collaborative Project
      Modern genomic sequencing tests often interrogate large numbers of genes. Identification of appropriate reference materials for development, validation studies, and quality assurance of these tests poses a significant challenge for laboratories. It is difficult to develop and maintain expert knowledge to identify all variants that must be validated to ensure analytic and clinical validity. Additionally, it is usually not possible to procure appropriate and characterized genomic DNA reference materials containing the number and scope of variants required. To address these challenges, the Centers for Disease Control and Prevention's Genetic Testing Reference Material Program (GeT-RM) has partnered with the Clinical Genome Resource (ClinGen) to develop a publicly available list of expert curated, clinically important variants. ClinGen Variant Curation Expert Panels nominated 546 variants found in 84 disease-associated genes, including common pathogenic and difficult-to-detect variants. Variant types nominated included 346 single nucleotide variants, 104 deletions, 37 copy number variants, 25 duplications, 18 deletion-insertions, 5 inversions, 4 insertions, 2 complex rearrangements, 3 difficult-to-sequence regions, and 2 fusions. This expert-curated variant list is a resource that provides a foundation for designing comprehensive validation studies and for creating in silico reference materials for clinical genomic test development and validation.
      Genetic testing has evolved from interrogating small sets of known, pathogenic variants in one or a few genes using targeted genotyping assays or Sanger sequencing to examining hundreds or thousands of genes at a time using next-generation sequencing (NGS). Large gene panels are now the norm in many clinical areas, such as hereditary cancer and cardiomyopathy, and are particularly useful for disorders with locus and allelic heterogeneity. Offerings from molecular testing laboratories range from single disorder to multidisorder panels, the latter often allowing molecular refinement of the initial clinical diagnosis (NIH, Genetic Testing Registry, https://www.ncbi.nlm.nih.gov/gtr, last accessed May 6, 2021).
      Although NGS has advanced many aspects of genetic testing, this technology has made the task of designing and developing analytically and clinically valid tests more complex. This process requires deep clinical and genetic expertise to understand the clinically relevant variant spectrum, and characterization of genomic context such as high GC content, highly homologous genes, or repetitive sequences pose challenges for most current NGS technologies.
      • Mandelker D.
      • Schmidt R.J.
      • Ankala A.
      • McDonald Gibson K.
      • Bowser M.
      • Sharma H.
      • Duffy E.
      • Hegde M.
      • Santani A.
      • Lebo M.
      • Funke B.
      Navigating highly homologous genes in a molecular diagnostic setting: a resource for clinical next-generation sequencing.
      ,
      • Bean L.J.H.
      • Funke B.
      • Carlston C.M.
      • Gannon J.L.
      • Kantarci S.
      • Krock B.L.
      • Zhang S.
      • Bayrak-Toydemir P.
      ACMG Laboratory Quality Assurance Committee
      Diagnostic gene sequencing panels: from design to report-a technical standard of the American College of Medical Genetics and Genomics (ACMG).
      When examining only one or a few well-studied genes, sufficient expertise to identify clinically important variants can be developed easily, but this aspect of test design and development does not scale to assays that examine hundreds or thousands of genes.
      • Rehm H.L.
      • Berg J.S.
      • Brooks L.D.
      • Bustamante C.D.
      • Evans J.P.
      • Landrum M.J.
      • Ledbetter D.H.
      • Maglott D.R.
      • Martin C.L.
      • Nussbaum R.L.
      • Plon S.E.
      • Ramos E.M.
      • Sherry S.T.
      • Watson M.S.
      ClinGen
      ClinGen--the clinical genome resource.
      Laboratories often cannot procure reference materials that encompass the scope of variants and variant types needed for NGS test development and validation because the supply of available characterized genomic DNA samples from cell lines or patient samples is limited and does not cover the range of genes and variants often included in NGS assays. Finally, traditional patient-derived reference materials, such as genomic DNA from cell lines, typically contain only one or two clinically important variants per sample, which can significantly increase the number of samples needed, and thus the complexity and cost of a validation study, when large numbers of variants need to be addressed.
      Proper analytical validation of genomic tests and large test panels require a combination of approaches to evaluate all components of the test, including the wet bench (eg, DNA extraction, library preparation, target enrichment), the sequencing platforms for generating raw reads, and the bioinformatics pipelines. To ensure optimal confidence in the ability of a given technology to detect variants within a genome or subregion (eg, exon), large numbers of variants for each clinically relevant type (single nucleotide variants, insertions, deletions, structural variants) need to be analyzed. In addition, test developers should demonstrate the test's ability to detect specific clinically important variants particularly when they are technically challenging (US Food and Drug Administration, https://www.fda.gov/media/99208/download, last accessed March 23, 2021).
      US Food and Drug Administration
      Considerations for design, development, and analytical validation of next generation sequencing (NGS) – based in vitro diagnostics (IVDs) intended to aid in the diagnosis of suspected germline diseases: guidance for stakeholders and Food and Drug Administration Staff. Rockville, MD, 2018.
      College of American Pathologists
      Molecular pathology 2020 checklist. Northfield, IL, 2020.
      Clinical and Laboratory Standards Institute
      MM17 Validation and Verification of Multiplex Nucleic Acid Assays.
      Different types of reference materials have been created to meet these needs.
      The 1000 Genomes project HapMap samples (1000 Genomes Project, https://www.internationalgenome.org/home, last accessed October 11, 2020) were the first generation of characterized genomes that were publicly available. More recently, the National Institute for Standards and Technology Genome in a Bottle project has characterized seven publicly available genomes including NA12878 and two son/mother/father trios of Ashkenazi Jewish and Han Chinese ancestry for use as reference materials (National Institute for Standards and Technology, https://www.nist.gov/programs-projects/genome-bottle, last accessed May 10, 2021).
      • Zook J.M.
      • Catoe D.
      • McDaniel J.
      • Vang L.
      • Spies N.
      • Sidow A.
      • et al.
      Extensive sequencing of seven human genomes to characterize benchmark reference materials.
      ,
      • Wang Y.-C.
      • Olson N.D.
      • Deikus G.
      • Shah H.
      • Wenger A.M.
      • Trow J.
      • Xiao C.
      • Sherry S.
      • Salit M.L.
      • Zook J.M.
      • Smith M.
      • Sebra R.
      High-coverage, long-read sequencing of Han Chinese trio reference samples.
      These samples have been extensively characterized using a number of different technologies and variant calling methods covering over 90% of the genome. The National Institute for Standards and Technology has generated publicly available, high-confidence benchmarking data sets that have been widely used by the clinical testing community and are now also covering structural variants, and variants in highly homologous regions.
      • Zook J.M.
      • McDaniel J.
      • Olson N.D.
      • Wagner J.
      • Parikh H.
      • Heaton H.
      • Irvine S.A.
      • Trigg L.
      • Truty R.
      • McLean C.Y.
      • De La Vega F.M.
      • Xiao C.
      • Sherry S.
      • Salit M.
      An open resource for accurately benchmarking small variant and reference calls.
      In addition, Illumina (San Diego, CA) produced a set of over 4.7 million phased variants from 17 samples of a large multigenerational pedigree.
      • Eberle M.A.
      • Fritzilas E.
      • Krusche P.
      • Källberg M.
      • Moore B.L.
      • Bekritsky M.A.
      • Iqbal Z.
      • Chuang H.-Y.
      • Humphray S.J.
      • Halpern A.L.
      • Kruglyak S.
      • Margulies E.H.
      • McVean G.
      • Bentley D.R.
      A reference data set of 5.4 million phased human variants validated by genetic inheritance from sequencing a three-generation 17-member pedigree.
      These data sets are excellent resources for establishing general analytical performance across various variant types, but do not meet the additional need to validate specific clinically important variants because the samples were generated from apparently healthy individuals and are unlikely to contain pathogenic variants in genes included in NGS panels. Other publicly available genomic DNA samples, such as those characterized by the Center for Disease Control and Prevention's Genetic Testing Reference Material Program (GeT-RM, https://www.cdc.gov/labquality/get-rm/index.html, last accessed May 21, 2021) and samples available from repositories such as the Coriell Institute for Medical Research (Camden, NJ) or the ATCC (Manassas, VA) have identified variants in some, but not all, genes likely to be included in NGS tests. Although useful, none of these genomic DNA samples, individually or as a group, are sufficient to represent the wide range of variants needed for a thorough and comprehensive validation of an NGS assay.
      As NGS tests have grown to include hundreds to thousands of genes, it has become technically and economically impractical to rely exclusively on DNA samples for assay development and validation studies to assess assay performance. Procuring sufficient reference materials containing specific variants has always been challenging and can be virtually impossible for large gene panels, particularly those that span multiple disorders. In addition, the commoditization of genomic testing and the availability of “off-the-shelf” gene panels have facilitated implementation of genomic tests. Well-established laboratories with longstanding clinical expertise often have archived DNA that can be used as reference materials for test development. By contrast, laboratories that do not already have in-depth expertise in the genomic profile of the disorders included in the panels are faced with a paucity of organized resources and guidance as to which clinically important variants require specific attention during test development. The Food and Drug Administration and professional organizations, including the College of American Pathologists, have begun to acknowledge these challenges and have responded with guidance that permits use of in silico reference materials to supplement clinical test validation and operation.
      US Food and Drug Administration
      Considerations for design, development, and analytical validation of next generation sequencing (NGS) – based in vitro diagnostics (IVDs) intended to aid in the diagnosis of suspected germline diseases: guidance for stakeholders and Food and Drug Administration Staff. Rockville, MD, 2018.
      ,
      College of American Pathologists
      Molecular pathology 2020 checklist. Northfield, IL, 2020.
      In 2019, the Clinical Laboratory Improvement Advisory Committee issued a recommendation to the Department of Health and Human Services that the GeT-RM program should develop in silico reference materials for NGS testing.
      Clinical Laboratory Improvement Advisory Committee
      April 10–11, 2018, summary report.
      As a first step toward this goal, GeT-RM has partnered with the Clinical Genome Resource (ClinGen, https://www.clinicalgenome.org, last accessed February 27, 2021) to develop lists of expert-curated, clinically important variants. This publication describes these variant lists, which can be used to design validation studies of the NGS informatics pipelines and will serve as a foundation for creating multivariant in silico reference materials by in silico mutagenesis of laboratory generated BAM or FASTQ files.
      • Duncavage E.J.
      • Abel H.J.
      • Pfeifer J.D.
      In silico proficiency testing for clinical next-generation sequencing.
      • Duncavage E.J.
      • Abel H.J.
      • Merker J.D.
      • Bodner J.B.
      • Zhao Q.
      • Voelkerding K.V.
      • Pfeifer J.D.
      A model study of in silico proficiency testing for clinical next-generation sequencing.
      • Li Z.
      • Fang S.
      • Zhang R.
      • Yu L.
      • Zhang J.
      • Bu D.
      • Sun L.
      • Zhao Y.
      • Li J.
      VarBen: generating in silico reference data sets for clinical next-generation sequencing bioinformatics pipeline evaluation.

      Materials and Methods

       Variant Nomination Process

      Members of 36 expert panels, including 35 of ClinGen's Variant Curation Expert Panels (VCEPs), and experts from the CFTR2 Project were contacted in March 2020 and asked to nominate clinically important variants in the genes covered by the expert panels. The Food and Drug Administration has recognized ClinGen's curation process and its resulting variant classifications as a regulatory-grade variant database.
      ClinGen
      FDA recognizes ClinGen assertions in clinvar - frequently asked questions. 2019.
      ,
      US Food and Drug Administration
      FDA takes new action to advance the development of reliable and beneficial genetic tests that can improve patient care. Rockville, MD: 2018.
      These VCEP-curated, Food and Drug Administration–recognized variants are available via NCBI's ClinVar database and ClinGen's Evidence Repository (https://erepo.clinicalgenome.org, last accessed May 10, 2021).
      Each expert panel was asked to nominate a maximum of 10 variants per disease area, including pathogenic variants that are the largest contributors to a disease as well as variants that may represent potential analytic challenges to laboratories. Panels submitted their variant nominations on a standardized template from April to June 2020. For each variant, the panels noted whether the basis for inclusion was: i) major contributor to disease, ii) analytic detection difficult, iii) filtration impact (may be inadvertently filtered out due to high allele frequency), or iv) other (with explanation provided in a comment box). The panels were instructed that any reason for difficult detection was appropriate, including variant type, homology issues, or a high allele frequency for a lower penetrant variant that might make the variant excluded by standard allele frequency filters. The ClinGen Allele Registry was used to standardize nomenclature for all nominated variants, and ClinVar Variation IDs and associated disorders were added where available. A full list of the expert panels that contributed variants can be found in the Acknowledgments, and the full list of nominated variants can be found in Supplemental Table S1.
      Not all of the ClinGen VCEPs that participated in this project have completed all four steps of the VCEP development process required for final approval by ClinGen.
      • Kalia S.S.
      • Adelman K.
      • Bale S.J.
      • Chung W.K.
      • Eng C.
      • Evans J.P.
      • Herman G.E.
      • Hufnagel S.B.
      • Klein T.E.
      • Korf B.R.
      • McKelvey K.D.
      • Ormond K.E.
      • Richards C.S.
      • Vlangos C.N.
      • Watson M.
      • Martin C.L.
      • Miller D.T.
      on behalf of the ACMG Secondary Findings Maintenance Working Group
      Recommendations for reporting of secondary findings in clinical exome and genome sequencing, 2016 update (ACMG SF v2.0): a policy statement of the American College of Medical Genetics and Genomics.
      ,
      ClinGen
      Guidelines for applying for variant or gene curation expert panel status. 2021.
      Of the variants submitted by the panels, 38% are from expert panels with final ClinGen approval to submit to ClinVar as “reviewed by expert panel,” whereas 62% are from ClinGen VCEPs that were still developing disease specifications for variant classification and were not yet fully ClinGen approved at the time of submission of variants to the project. VCEP status at the time of manuscript submission is shown in Supplemental Table S1. Current status can be found on the ClinGen website.
      ClinGen
      Clinical domain working groups.

       Limitations of Variant Selection Process

      Several limitations of the variant selection process should be noted. There were no explicit requirements given to the expert panels in terms of the definitions of categories i, ii, and iii, and therefore, each may have used slightly different definitions to select variants to nominate for this project. In addition, some expert panels may have used laboratory data that are not in the public domain to identify nominated variants, and this information was not requested during the submission process. Furthermore, some expert panels represent genes and diseases that are very rare, and therefore, variants may not, in general, be highly prevalent. These are not considered serious limitations because these are first-generation variant lists, which provide an entirely new resource. It is expected that this effort will continue in years to come to broaden and deepen this resource.

      Results

      Nominations of clinically important pathogenic or difficult-to-detect variants were received from 24 expert panels of 36 contacted. Overall, 546 unique variants in 84 genes were nominated (Supplemental Table S1) including 29 of the 73 genes recommended by the American College of Medical Genetics and Genomics for reporting of incidental or secondary findings (Supplemental Table S2), which are included as part of virtually all proactive genomic health tests.
      • Kalia S.S.
      • Adelman K.
      • Bale S.J.
      • Chung W.K.
      • Eng C.
      • Evans J.P.
      • Herman G.E.
      • Hufnagel S.B.
      • Klein T.E.
      • Korf B.R.
      • McKelvey K.D.
      • Ormond K.E.
      • Richards C.S.
      • Vlangos C.N.
      • Watson M.
      • Martin C.L.
      • Miller D.T.
      on behalf of the ACMG Secondary Findings Maintenance Working Group
      Recommendations for reporting of secondary findings in clinical exome and genome sequencing, 2016 update (ACMG SF v2.0): a policy statement of the American College of Medical Genetics and Genomics.
      ,
      • Miller D.T.
      • Lee K.
      • Chung W.K.
      • Gordon A.S.
      • Herman G.E.
      • Klein T.E.
      • Stewart D.R.
      • Amendola L.M.
      • Adelman K.
      • Bale S.J.
      • Gollob M.H.
      • Harrison S.M.
      • Hershberger R.E.
      • McKelvey K.
      • Richards C.S.
      • Vlangos C.N.
      • Watson M.S.
      • Martin C.L.
      ACMG Secondary Findings Working Group
      ACMG SF v3.0 list for reporting of secondary findings in clinical exome and genome sequencing: a policy statement of the American College of Medical Genetics and Genomics (ACMG).
      The nominated variants are causative for a wide range of diseases, many of which are commonly tested by NGS, including heritable cancers, inborn errors of metabolism, cardiomyopathy, diabetes, and immune disorders (Supplemental Table S2). A number of different variant types were nominated, including 346 single nucleotide variants, 104 deletions, 37 copy number variants, 25 duplications, 18 deletion-insertions, 5 inversions, 4 insertions, 2 complex rearrangements, 2 fusions, and 3 variants in regions defined as "a difficult region to cover and map variants." The variants were nominated for a variety of reasons: 355 are major contributors to disease, 111 are analytically difficult to detect, 28 may have filtration impacts (be inadvertently filtered out due to high allele frequency), 6 had multiple reasons for inclusion, and 46 were listed as other or no explanation was provided (Supplemental Table S1).

      Discussion

      Clinical laboratories in the United States are required by regulation and guided by professional or best practice standards to use characterized reference materials for test development, validation and verification studies, quality control, and proficiency testing.
      College of American Pathologists
      Molecular pathology 2020 checklist. Northfield, IL, 2020.
      ,
      • Chen B.
      • Gagnon M.
      • Shahangian S.
      • Anderson N.L.
      • Howerton D.A.
      • Boone J.D.
      Centers for Disease Control and Prevention
      Good laboratory practices for molecular genetic testing for heritable diseases and conditions.
      Association for Molecular Pathology statement. Recommendations for in-house development and operation of molecular diagnostic tests.
      International Organisation for Standardization's Technical Committee 212
      ISO 15189 Medical Laboratories-Requirements for Quality and Competence.
      The Clinical Laboratory
      Improvement Amendments. Code of Federal Regulations. Title 42, Chapter IV, Subchapter G.
      American College of Medical Genetics
      Standards and guidelines for clinical genetics laboratories. 2008 ed. 2007.
      Washington State Legislature
      WAC 246-338-090. Quality control. 2019.
      New York State Department of Health
      Clinical laboratory evaluation program.
      Certain aspects of test development and validation, such as DNA extraction, library preparation, and sequence generation will always require the use of genomic DNA from cell lines or patient samples. However, given the scope of many NGS tests, it is no longer feasible to rely solely upon genomic DNA samples containing only one or a few variants of interest to validate an assay that tests an entire exome or genome or many genes within a large NGS panel, and novel types of reference materials are needed to enable assessment of a test's analytical validity and establish adequate clinical validity. In silico reference materials can be a useful tool to supplement the use of genomic DNA reference materials for the development and validation of NGS bioinformatic pipelines.
      The Association for Molecular Pathology and the College of American Pathologists allow the use of in silico reference materials to supplement genomic DNA for the validation of bioinformatic pipelines.
      College of American Pathologists
      Molecular pathology 2020 checklist. Northfield, IL, 2020.
      ,
      Clinical and Laboratory Standards Institute
      MM17 Validation and Verification of Multiplex Nucleic Acid Assays.
      ,
      • Aziz N.
      • Zhao Q.
      • Bry L.
      • Driscoll D.K.
      • Funke B.
      • Gibson J.S.
      • Grody W.W.
      • Hegde M.R.
      • Hoeltge G.A.
      • Leonard D.G.B.
      • Merker J.D.
      • Nagarajan R.
      • Palicki L.A.
      • Robetorye R.S.
      • Schrijver I.
      • Weck K.E.
      • Voelkerding K.V.
      College of American Pathologists' laboratory standards for next-generation sequencing clinical tests.
      • Roy S.
      • Coldren C.
      • Karunamurthy A.
      • Kip N.S.
      • Klee E.W.
      • Lincoln S.E.
      • Leon A.
      • Pullambhatla M.
      • Temple-Smolkin R.L.
      • Voelkerding K.V.
      • Wang C.
      • Carter A.B.
      Standards and guidelines for validating next-generation sequencing bioinformatics pipelines: a joint recommendation of the Association for Molecular Pathology and the College of American Pathologists.
      • Santani A.
      • Simen B.B.
      • Briggs M.
      • Lebo M.
      • Merker J.D.
      • Nikiforova M.
      • Vasalos P.
      • Voelkerding K.
      • Pfeifer J.
      • Funke B.
      Designing and implementing NGS tests for inherited disorders: a practical framework with step-by-step guidance for clinical laboratories.
      Specialized software
      • Duncavage E.J.
      • Abel H.J.
      • Pfeifer J.D.
      In silico proficiency testing for clinical next-generation sequencing.
      ,
      • Duncavage E.J.
      • Abel H.J.
      • Merker J.D.
      • Bodner J.B.
      • Zhao Q.
      • Voelkerding K.V.
      • Pfeifer J.D.
      A model study of in silico proficiency testing for clinical next-generation sequencing.
      ,
      • Patil S.A.
      • Mujacic I.
      • Ritterhouse L.L.
      • Segal J.P.
      • Kadri S.
      insiM: in silico Mutator software for bioinformatics pipeline validation of clinical next-generation sequencing assays.
      has been developed to edit sequence variants into BAM or FASTQ files generated by the assay being validated to create customized reference materials. The software is publicly available, and several companies offer this service to laboratories. The GeT-RM program is planning to use these curated variants and file editing software to pilot development and use of in silico reference materials in clinical laboratories.
      The number of variants that can be added is potentially unlimited, but care must be taken to avoid adding too many variants in cis to the same sequence reads because it may affect alignment to the reference sequence. The types of variants that can be introduced remains limited to single nucleotide variants and smaller insertions and deletions at different allele fractions, although the ability to introduce copy number variants is currently being developed. This approach has been used successfully by the College of American Pathologists for its NGS proficiency testing challenges.
      • Duncavage E.J.
      • Abel H.J.
      • Pfeifer J.D.
      In silico proficiency testing for clinical next-generation sequencing.
      ,
      • Duncavage E.J.
      • Abel H.J.
      • Merker J.D.
      • Bodner J.B.
      • Zhao Q.
      • Voelkerding K.V.
      • Pfeifer J.D.
      A model study of in silico proficiency testing for clinical next-generation sequencing.
      The variant list developed by the effort presented here is available on the GeT-RM and ClinGen websites and can serve as a knowledge resource to help laboratories identify important pathogenic and difficult-to-detect variants in genes that are included in their assays, as well as a foundation for generating customized in silico and synthetic DNA reference materials. GeT-RM and ClinGen will continue to collaborate to add to the current variant list and invite input from the genetics community about this list and the processes used to generate it.

      Acknowledgments

      We thank the participating ClinGen VCEP/Expert Panels and contributors: Troy R. Torgerson, Ivan Chinn, Roshini S. Abraham, Mikko Seppanen, Anne Puel, Sergio Rosenzweig, Yu-Lung Lau, Antonio Condino-Neto, Rebecca Marsh, Miao Sun, Craig Platt, Kathleen Sullivan, Manish Butte, Isabelle Meyts, Janna Saarela, Britt Johnson, Joe Jacher, Siobhan Burns, Capucine Picard, Ryan Martinez, and Brooke C. Palus (Immunology CDWG); Jessica Mester, Madhuri Hegde, and Charis Eng (PTEN VCEP); Claude Houdayer, Mads Thomassen, Arjen Mensenkamp, Erik Teugels, Michael Parsons, Inge Sokilde Pedersen, and Amanda Spurdle (ENIGMA BRCA1 and BRCA2 VCEP); Toni Pollin, Ruth Cosentino, Kristin Maloney, Haichen Zhang, Kevin Colclough, Glenn Maston, Martina Škopková, Milena Teles, Lucas Santana, Petra Dušátková, and Fabrizio Barbetti (Monogenic Diabetes VCEP); Marcy Richardson and William Foulkes (Hereditary Breast, Ovarian, and Pancreatic Cancer VCEP); Megan Frone, Douglas Stewart, Leora Witkowski, William Foulkes, and Ann Carr (DICER1 and miRNA-Processing Gene VCEP); Ozge Birsoy, Carsten Bonnemann, and Brooke C. Palus (Congenital Myopathies VCEP); Megan Frone, Sharon Savage, and Jessica Mester (TP53 VCEP); Xi Luo and Sharon Plon (CDH1 VCEP); Catherine Rehder and Jenny Goldstein (Lysosomal Storage Disorders VCEP); Karen Raraigh and Garry Cutting (CFTR2); Desiree DeMille, Pinar Bayrak-Toydemir, and Jamie Kamm-McDonald (Hereditary Hemorrhagic Telangiectasia VCEP); Birgit Funke, Melissa A. Kelly, Hana Zouk, and Kate L. Thomson (Cardiomyopathy VCEP); Jennifer Johnson and Leslie Biesecker (Malignant Hyperthermia Susceptibility VCEP); Hannah Wand, Mafalda Bourbon, Joana Chora, and Josh Knowles (Familial Hypercholesterolemia VCEP); Cindy James, J. Peter van Tintelen, Jan D.H. Jongbloed, Brittney Murray, and Crystal Tichnell (Arrhythmogenic Right Ventricular Cardiomyopathy VCEP); Raymond Kim, Sean Delong, Andreea Chiorean, Kristen Farncombe, and Deborah Ritter (VHL VCEP); Petros Kountouris, Marina Kleanthous, Cornelis L. Harteveld, Celeste Bento, and Joanne Traeger-Synodinos (Hemoglobinopathy VCEP); Julie De Backer, Laura Muiño Mosquera, and Paul Coucke (FBN1 VCEP); Stefan Aretz, Isabel Spier, Ian Frayling, John-Paul Plazzer, Deborah Ritter, and Andreas Laner (InSiGHT Hereditary Colorectal Cancer/Polyposis VCEP); Rong Mao, May Jasmine Flowers, and Meredith Weaver (ACADVL VCEP); Meredith Weaver, Diane Zastrow, and William Craigen (Phenylketonuria VCEP); Emmanuelle Souzeau, Andrew Dubowsky, Janey Wiggs, Jamie Craig, Kathryn Burdon, Alex Hewitt, David Mackey, Owen Siggs, Francesca Pasutto, Terri Young, John Fingert, and Erin Boese (Glaucoma VCEP); Danielle Azzariti, Martin Zenker, Helene Cave, and Lisa Vincent (RASopathy VCEP).

      Supplemental Data

      References

        • Mandelker D.
        • Schmidt R.J.
        • Ankala A.
        • McDonald Gibson K.
        • Bowser M.
        • Sharma H.
        • Duffy E.
        • Hegde M.
        • Santani A.
        • Lebo M.
        • Funke B.
        Navigating highly homologous genes in a molecular diagnostic setting: a resource for clinical next-generation sequencing.
        Genet Med. 2016; 18: 1282-1289
        • Bean L.J.H.
        • Funke B.
        • Carlston C.M.
        • Gannon J.L.
        • Kantarci S.
        • Krock B.L.
        • Zhang S.
        • Bayrak-Toydemir P.
        • ACMG Laboratory Quality Assurance Committee
        Diagnostic gene sequencing panels: from design to report-a technical standard of the American College of Medical Genetics and Genomics (ACMG).
        Genet Med. 2020; 22: 453-461
        • Rehm H.L.
        • Berg J.S.
        • Brooks L.D.
        • Bustamante C.D.
        • Evans J.P.
        • Landrum M.J.
        • Ledbetter D.H.
        • Maglott D.R.
        • Martin C.L.
        • Nussbaum R.L.
        • Plon S.E.
        • Ramos E.M.
        • Sherry S.T.
        • Watson M.S.
        • ClinGen
        ClinGen--the clinical genome resource.
        N Engl J Med. 2015; 372: 2235-2242
        • US Food and Drug Administration
        Considerations for design, development, and analytical validation of next generation sequencing (NGS) – based in vitro diagnostics (IVDs) intended to aid in the diagnosis of suspected germline diseases: guidance for stakeholders and Food and Drug Administration Staff. Rockville, MD, 2018.
        (Available at)
        https://www.fda.gov/media/99208/download
        Date accessed: March 23, 2021
        • College of American Pathologists
        Molecular pathology 2020 checklist. Northfield, IL, 2020.
        (Available at)
        https://www.cap.org
        Date accessed: March 23, 2021
        • Clinical and Laboratory Standards Institute
        MM17 Validation and Verification of Multiplex Nucleic Acid Assays.
        ed 2. Clinical and Laboratory Standards Institute, Wayne, PA2018
        • Zook J.M.
        • Catoe D.
        • McDaniel J.
        • Vang L.
        • Spies N.
        • Sidow A.
        • et al.
        Extensive sequencing of seven human genomes to characterize benchmark reference materials.
        Sci Data. 2016; 3: 160025
        • Wang Y.-C.
        • Olson N.D.
        • Deikus G.
        • Shah H.
        • Wenger A.M.
        • Trow J.
        • Xiao C.
        • Sherry S.
        • Salit M.L.
        • Zook J.M.
        • Smith M.
        • Sebra R.
        High-coverage, long-read sequencing of Han Chinese trio reference samples.
        Sci Data. 2019; 6: 91
        • Zook J.M.
        • McDaniel J.
        • Olson N.D.
        • Wagner J.
        • Parikh H.
        • Heaton H.
        • Irvine S.A.
        • Trigg L.
        • Truty R.
        • McLean C.Y.
        • De La Vega F.M.
        • Xiao C.
        • Sherry S.
        • Salit M.
        An open resource for accurately benchmarking small variant and reference calls.
        Nat Biotechnol. 2019; 37: 561-566
        • Eberle M.A.
        • Fritzilas E.
        • Krusche P.
        • Källberg M.
        • Moore B.L.
        • Bekritsky M.A.
        • Iqbal Z.
        • Chuang H.-Y.
        • Humphray S.J.
        • Halpern A.L.
        • Kruglyak S.
        • Margulies E.H.
        • McVean G.
        • Bentley D.R.
        A reference data set of 5.4 million phased human variants validated by genetic inheritance from sequencing a three-generation 17-member pedigree.
        Genome Res. 2017; 27: 157-164
        • Clinical Laboratory Improvement Advisory Committee
        April 10–11, 2018, summary report.
        US Department of Health and Human Services, Baltimore, MD2019 (Available at) (accessed June 15, 2021)
        • Duncavage E.J.
        • Abel H.J.
        • Pfeifer J.D.
        In silico proficiency testing for clinical next-generation sequencing.
        J Mol Diagn. 2017; 19: 35-42
        • Duncavage E.J.
        • Abel H.J.
        • Merker J.D.
        • Bodner J.B.
        • Zhao Q.
        • Voelkerding K.V.
        • Pfeifer J.D.
        A model study of in silico proficiency testing for clinical next-generation sequencing.
        Arch Pathol Lab Med. 2016; 140: 1085-1091
        • Li Z.
        • Fang S.
        • Zhang R.
        • Yu L.
        • Zhang J.
        • Bu D.
        • Sun L.
        • Zhao Y.
        • Li J.
        VarBen: generating in silico reference data sets for clinical next-generation sequencing bioinformatics pipeline evaluation.
        J Mol Diagn. 2021; 23: 285-299
        • ClinGen
        FDA recognizes ClinGen assertions in clinvar - frequently asked questions. 2019.
        (Available at)
        • US Food and Drug Administration
        FDA takes new action to advance the development of reliable and beneficial genetic tests that can improve patient care. Rockville, MD: 2018.
        (Available at)
        • Kalia S.S.
        • Adelman K.
        • Bale S.J.
        • Chung W.K.
        • Eng C.
        • Evans J.P.
        • Herman G.E.
        • Hufnagel S.B.
        • Klein T.E.
        • Korf B.R.
        • McKelvey K.D.
        • Ormond K.E.
        • Richards C.S.
        • Vlangos C.N.
        • Watson M.
        • Martin C.L.
        • Miller D.T.
        • on behalf of the ACMG Secondary Findings Maintenance Working Group
        Recommendations for reporting of secondary findings in clinical exome and genome sequencing, 2016 update (ACMG SF v2.0): a policy statement of the American College of Medical Genetics and Genomics.
        Genet Med. 2017; 19: 249-255
        • ClinGen
        Guidelines for applying for variant or gene curation expert panel status. 2021.
        (Available at)
        • ClinGen
        Clinical domain working groups.
        (Available at)
        • Miller D.T.
        • Lee K.
        • Chung W.K.
        • Gordon A.S.
        • Herman G.E.
        • Klein T.E.
        • Stewart D.R.
        • Amendola L.M.
        • Adelman K.
        • Bale S.J.
        • Gollob M.H.
        • Harrison S.M.
        • Hershberger R.E.
        • McKelvey K.
        • Richards C.S.
        • Vlangos C.N.
        • Watson M.S.
        • Martin C.L.
        • ACMG Secondary Findings Working Group
        ACMG SF v3.0 list for reporting of secondary findings in clinical exome and genome sequencing: a policy statement of the American College of Medical Genetics and Genomics (ACMG).
        Genet Med. 2021; 23: 1381-1390
        • Chen B.
        • Gagnon M.
        • Shahangian S.
        • Anderson N.L.
        • Howerton D.A.
        • Boone J.D.
        • Centers for Disease Control and Prevention
        Good laboratory practices for molecular genetic testing for heritable diseases and conditions.
        MMWR Recomm Rep. 2009; 58 (quiz CE-31-34): 1-37
      1. Association for Molecular Pathology statement. Recommendations for in-house development and operation of molecular diagnostic tests.
        Am J Clin Pathol. 1999; 111: 449-463
        • International Organisation for Standardization's Technical Committee 212
        ISO 15189 Medical Laboratories-Requirements for Quality and Competence.
        International Organisation for Standardization, Geneva, Switzerland2012
        • The Clinical Laboratory
        Improvement Amendments. Code of Federal Regulations. Title 42, Chapter IV, Subchapter G.
        Part 493. 1988;
        • American College of Medical Genetics
        Standards and guidelines for clinical genetics laboratories. 2008 ed. 2007.
        (Available at)
        • Washington State Legislature
        WAC 246-338-090. Quality control. 2019.
        (Available at)
        • New York State Department of Health
        Clinical laboratory evaluation program.
        (Available at)
        http://www.wadsworth.org/clep
        Date accessed: May 21, 2021
        • Aziz N.
        • Zhao Q.
        • Bry L.
        • Driscoll D.K.
        • Funke B.
        • Gibson J.S.
        • Grody W.W.
        • Hegde M.R.
        • Hoeltge G.A.
        • Leonard D.G.B.
        • Merker J.D.
        • Nagarajan R.
        • Palicki L.A.
        • Robetorye R.S.
        • Schrijver I.
        • Weck K.E.
        • Voelkerding K.V.
        College of American Pathologists' laboratory standards for next-generation sequencing clinical tests.
        Arch Pathol Lab Med. 2015; 139: 481-493
        • Roy S.
        • Coldren C.
        • Karunamurthy A.
        • Kip N.S.
        • Klee E.W.
        • Lincoln S.E.
        • Leon A.
        • Pullambhatla M.
        • Temple-Smolkin R.L.
        • Voelkerding K.V.
        • Wang C.
        • Carter A.B.
        Standards and guidelines for validating next-generation sequencing bioinformatics pipelines: a joint recommendation of the Association for Molecular Pathology and the College of American Pathologists.
        J Mol Diagn. 2018; 20: 4-27
        • Santani A.
        • Simen B.B.
        • Briggs M.
        • Lebo M.
        • Merker J.D.
        • Nikiforova M.
        • Vasalos P.
        • Voelkerding K.
        • Pfeifer J.
        • Funke B.
        Designing and implementing NGS tests for inherited disorders: a practical framework with step-by-step guidance for clinical laboratories.
        J Mol Diagn. 2019; 21: 369-374
        • Patil S.A.
        • Mujacic I.
        • Ritterhouse L.L.
        • Segal J.P.
        • Kadri S.
        insiM: in silico Mutator software for bioinformatics pipeline validation of clinical next-generation sequencing assays.
        J Mol Diagn. 2019; 21: 19-26