Browsing by Subject "Reproducibility"

Sort by: Order: Results:

Now showing items 1-13 of 13
  • Varga, Zsuzsanna; Lebeau, Annette; Bu, Hong; Hartmann, Arndt; Penault-Llorca, Frederique; Guerini-Rocco, Elena; Schraml, Peter; Symmans, Fraser; Stoehr, Robert; Teng, Xiaodong; Turzynski, Andreas; von Wasielewski, Reinhard; Guertler, Claudia; Laible, Mark; Schlombs, Kornelia; Joensuu, Heikki; Keller, Thomas; Sinn, Peter; Sahin, Ugur; Bartlett, John; Viale, Giuseppe (2017)
    Background: Accurate determination of the predictive markers human epidermal growth factor receptor 2 (HER2/ERBB2), estrogen receptor (ER/ESR1), progesterone receptor (PgR/PGR), and marker of proliferation Ki67 (MKI67) is indispensable for therapeutic decision making in early breast cancer. In this multicenter prospective study, we addressed the issue of inter- and intrasite reproducibility using the recently developed reverse transcription-quantitative real-time polymerase chain reaction-based MammaTyper (R) test. Methods: Ten international pathology institutions participated in this study and determined messenger RNA expression levels of ERBB2, ESR1, PGR, and MKI67 in both centrally and locally extracted RNA from formalin-fixed, paraffin-embedded breast cancer specimens with the MammaTyper (R) test. Samples were measured repeatedly on different days within the local laboratories, and reproducibility was assessed by means of variance component analysis, Fleiss' kappa statistics, and interclass correlation coefficients (ICCs). Results: Total variations in measurements of centrally and locally prepared RNA extracts were comparable; therefore, statistical analyses were performed on the complete dataset. Intersite reproducibility showed total SDs between 0.21 and 0.44 for the quantitative single-marker assessments, resulting in ICC values of 0.980-0.998, demonstrating excellent agreement of quantitative measurements. Also, the reproducibility of binary single-marker results (positive/negative), as well as the molecular subtype agreement, was almost perfect with kappa values ranging from 0.90 to 1.00. Conclusions: On the basis of these data, the MammaTyper (R) has the potential to substantially improve the current standards of breast cancer diagnostics by providing a highly precise and reproducible quantitative assessment of the established breast cancer biomarkers and molecular subtypes in a decentralized workup.
  • Gong, BS; Li, D; Kusko, R; Novoradovskaya, N; Zhang, YF; Wang, SZ; Pabon-Pena, C; Zhang, ZH; Lai, K; Cai, WS; LoCoco, JS; Lader, E; Richmond, TA; Mittal, VK; Liu, LC; Johann, DJ; Willey, JC; Bushel, PR; Yu, Y; Xu, C; Chen, GC; Burgess, D; Cawley, S; Giorda, K; Haseley, N; Qiu, FJ; Wilkins, K; Arib, H; Attwooll, C; Babson, K; Bao, LL; Bao, WJ; Lucas, AB; Best, H; Bhandari, A; Bisgin, H; Blackburn, J; Blomquist, TM; Boardman, L; Burgher, B; Butler, DJ; Chang, CJ; Chaubey, A; Chen, T; Chierici, M; Chin, CR; Close, D; Conroy, J; Coleman, JC; Craig, DJ; Crawford, E; del Pozo, A; Deveson, IW; Duncan, D; Eterovic, AK; Fan, XH; Foox, J; Furlanello, C; Ghosal, A; Glenn, S; Guan, MJ; Haag, C; Hang, XY; Happe, S; Hennigan, B; Hipp, J; Hong, HX; Horvath, K; Hu, JH; Hung, LY; Jarosz, M; Kerkhof, J; Kipp, B; Kreil, DP; Lapunzina, P; Li, P; Li, QZ; Li, WH; Li, ZG; Liang, Y; Liu, SQ; Liu, ZC; Ma, C; Marella, N; Martin-Arenas, R; Megherbi, DB; Meng, QC; Mieczkowski, PA; Morrison, T; Muzny, D; Ning, BT; Parsons, BL; Paweletz, CP; Pirooznia, M; Qu, WB; Raymond, A; Rindler, P; Ringler, R; Sadikovic, B; Scherer, A; Schulze, E; Sebra, R; Shaknovich, R; Shi, Q; Shi, TL; Silla-Castro, JC; Smith, M; Lopez, MS; Song, P; Stetson, D; Strahl, M; Stuart, A; Supplee, J; Szankasi, P; Tan, HW; Tang, LY; Tao, YH; Thakkar, S; Thierry-Mieg, D; Thierry-Mieg, J; Thodima, VJ; Thomas, D; Tichy, B; Tom, N; Garcia, EV; Verma, S; Walker, K; Wang, C; Wang, JW; Wang, YX; Wen, ZN; Wirta, V; Wu, LH; Xiao, CL; Xiao, WZ; Xu, SB; Yang, M; Ying, JM; Yip, SH; Zhang, GL; Zhang, S; Zhao, MR; Zheng, YT; Zhou, XY; Mason, CE; Mercer, T; Tong, WD; Shi, LM; Jones, W; Xu, JS (2021)
    BackgroundTargeted sequencing using oncopanels requires comprehensive assessments of accuracy and detection sensitivity to ensure analytical validity. By employing reference materials characterized by the U.S. Food and Drug Administration-led SEquence Quality Control project phase2 (SEQC2) effort, we perform a cross-platform multi-lab evaluation of eight Pan-Cancer panels to assess best practices for oncopanel sequencing.ResultsAll panels demonstrate high sensitivity across targeted high-confidence coding regions and variant types for the variants previously verified to have variant allele frequency (VAF) in the 5-20% range. Sensitivity is reduced by utilizing VAF thresholds due to inherent variability in VAF measurements. Enforcing a VAF threshold for reporting has a positive impact on reducing false positive calls. Importantly, the false positive rate is found to be significantly higher outside the high-confidence coding regions, resulting in lower reproducibility. Thus, region restriction and VAF thresholds lead to low relative technical variability in estimating promising biomarkers and tumor mutational burden.ConclusionThis comprehensive study provides actionable guidelines for oncopanel sequencing and clear evidence that supports a simplified approach to assess the analytical performance of oncopanels. It will facilitate the rapid implementation, validation, and quality control of oncopanels in clinical use.
  • Allkja, Jontana; Bjarnsholt, Thomas; Coenye, Tom; Cos, Paul; Fallarero, Adyary; Harrison, Joe J.; Lopes, Susana P.; Oliver, Antonio; Pereira, Maria Olivia; Ramage, Gordon; Shirtliff, Mark E.; Stoodley, Paul; Webb, Jeremy S.; Zaat, Sebastian A. J.; Goeres, Darla M.; Azevedo, Nuno Filipe (2020)
    The lack of reproducibility of published studies is one of the major issues facing the scientific community, and the field of biofilm microbiology has been no exception. One effective strategy against this multifaceted problem is the use of minimum information guidelines. This strategy provides a guide for authors and reviewers on the necessary information that a manuscript should include for the experiments in a study to be clearly interpreted and independently reproduced. As a result of several discussions between international groups working in the area of biofilms, we present a guideline for the spectrophotometric and fluorometric assessment of biofilm formation in microplates. This guideline has been divided into 5 main sections, each presenting a comprehensive set of recommendations. The intention of the minimum information guideline is to improve the quality of scientific communication that will augment interlaboratory reproducibility in biofilm microplate assays.
  • Hugenholtz, Floor; de Vos, Willem M. (2018)
    Since the early days of the intestinal microbiota research, mouse models have been used frequently to study the interaction of microbes with their host. However, to translate the knowledge gained from mouse studies to a human situation, the major spatio-temporal similarities and differences between intestinal microbiota in mice and humans need to be considered. This is done here with specific attention for the comparative physiology of the intestinal tract, the effect of dietary patterns and differences in genetics. Detailed phylogenetic and metagenomic analysis showed that while many common genera are found in the human and murine intestine, these differ strongly in abundance and in total only 4% of the bacterial genes are found to share considerable identity. Moreover, a large variety of murine strains is available yet most of the microbiota research is performed in wild-type, inbred strains and their transgenic derivatives. It has become increasingly clear that the providers, rearing facilities and the genetic background of these mice have a significant impact on the microbial composition and this is illustrated with recent experimental data. This may affect the reproducibility of mouse microbiota studies and their conclusions. Hence, future studies should take these into account to truly show the effect of diet, genotype or environmental factors on the microbial composition.
  • Brandstack, Nina; Kurki, T.; Laalo, J.; Kauko, T.; Tenovuo, O. (2016)
    Reproducibility of two different methods for quantifying fiber tracts by using a diffusion tensor imaging (DTI) sequence suitable for clinical magnetic resonance imaging (MRI) protocols was evaluated. DTI of 15 subjects was used to analyze intra-rater and inter-rater reproducibility. Another 10 subjects underwent MRI twice for assessment of between-scan reliability. Ten long association tracts were defined by fiber tracking using inclusion and exclusion regions of interest (ROIs). Whole-tract analysis and tractography-based core analysis were performed, and the effect of fractional anisotropy (FA 0.15/0.30) and turning angle threshold (27A degrees/60A degrees) on reproducibility was evaluated. Additionally, ROI measurements were performed in the core of the tracts. For the tract-based methods, intra-rater and inter-rater reliabilities of FA and mean diffusivity (MD) measurements were excellent. Between-scan reproducibility was good or excellent in 127 of 130 of the measurements. There was no systematic difference in the reproducibility of the FA, MD, and volume measurements depending on the FA or turning angle threshold. For the cross-sectional ROI measurements, reliability showed large variation from poor to excellent depending on the tract. Compared with the commonly used cross-sectional core ROI method, the tract-based analyses seem to be a more robust way to identify and measure white matter tracts of interest, and provide a novel reproducible tool to perform core analysis.
  • Rosenblatt, Alana J; Lappalainen, Anu K.; James, Nina A; Webster, Natalie S L; Caraguel, Charles G B (BioMed Central, 2018)
    Abstract Background The Dachshund is a chondrodystrophic breed of dog predisposed to premature degeneration and calcification, and subsequent herniation, of intervertebral discs (IVDs). This condition is heritable in Dachshunds and breeding candidates are screened for radiographically detectable intervertebral disc calcification (RDIDC), a feature of advanced disc degeneration and a prognostic factor for clinical disease. RDIDC scoring has been previously shown to be consistent within scorers; however, strong scorer effect (subjectivity) was also reported. The aim of this study was to estimate the within- and between-scorer agreement (repeatability and reproducibility, respectively) of computed tomography (CT) scanning and magnetic resonance imaging (MRI) for scoring IVD calcification, and to compare these modalities with radiographic scoring. Results Twenty-one Dachshund dogs were screened for IVD calcification using the three imaging modalities. Three scorers scored each case twice, independently. Repeatability was highest for radiography (95.4%), and significantly higher than for CT (90.4%) but not MRI (93.8%). Reproducibility was also highest for radiography (92.9%), but not significantly higher than for CT or MRI (89.4% and 86.4%, respectively). Overall, CT scored IVDs differently than radiography and MRI (64.8% and 62.7% agreement, respectively), while radiography and MRI scored more similarly (85.7% agreement). Conclusions Despite high precision for radiography, previous evidence of scorer subjectivity was confirmed, which was not generally observed with CT and MRI. The increased consistency of radiography may be related to prior scorer experience with the modality and RDIDC scoring. This study does not support replacing radiography with CT or MRI to screen for heritable IVD calcification in breeding Dachshunds; however, evaluation of dog-level precision and the accuracy of each modality is recommended.
  • Rosenblatt, Alana Jayne; Lappalainen, Anu Katriina; James, Nina Alice; Webster, Natalie Siu Ling; Caraguel, Charles Gregoire Benedict (2018)
    BackgroundThe Dachshund is a chondrodystrophic breed of dog predisposed to premature degeneration and calcification, and subsequent herniation, of intervertebral discs (IVDs). This condition is heritable in Dachshunds and breeding candidates are screened for radiographically detectable intervertebral disc calcification (RDIDC), a feature of advanced disc degeneration and a prognostic factor for clinical disease. RDIDC scoring has been previously shown to be consistent within scorers; however, strong scorer effect (subjectivity) was also reported. The aim of this study was to estimate the within- and between-scorer agreement (repeatability and reproducibility, respectively) of computed tomography (CT) scanning and magnetic resonance imaging (MRI) for scoring IVD calcification, and to compare these modalities with radiographic scoring.ResultsTwenty-one Dachshund dogs were screened for IVD calcification using the three imaging modalities. Three scorers scored each case twice, independently. Repeatability was highest for radiography (95.4%), and significantly higher than for CT (90.4%) but not MRI (93.8%). Reproducibility was also highest for radiography (92.9%), but not significantly higher than for CT or MRI (89.4% and 86.4%, respectively). Overall, CT scored IVDs differently than radiography and MRI (64.8% and 62.7% agreement, respectively), while radiography and MRI scored more similarly (85.7% agreement).ConclusionsDespite high precision for radiography, previous evidence of scorer subjectivity was confirmed, which was not generally observed with CT and MRI. The increased consistency of radiography may be related to prior scorer experience with the modality and RDIDC scoring. This study does not support replacing radiography with CT or MRI to screen for heritable IVD calcification in breeding Dachshunds; however, evaluation of dog-level precision and the accuracy of each modality is recommended.
  • Jaiswal, Alok; Peddinti, Gopal; Akimov, Yevhen; Wennerberg, Krister; Kuznetsov, Sergey; Tang, Jing; Aittokallio, Tero (2017)
    Background: Genome-wide loss-of-function profiling is widely used for systematic identification of genetic dependencies in cancer cells; however, the poor reproducibility of RNA interference (RNAi) screens has been a major concern due to frequent off-target effects. Currently, a detailed understanding of the key factors contributing to the sub-optimal consistency is still a lacking, especially on how to improve the reliability of future RNAi screens by controlling for factors that determine their off-target propensity. Methods: We performed a systematic, quantitative analysis of the consistency between two genome-wide shRNA screens conducted on a compendium of cancer cell lines, and also compared several gene summarization methods for inferring gene essentiality from shRNA level data. We then devised novel concepts of seed essentiality and shRNA family, based on seed region sequences of shRNAs, to study in-depth the contribution of seed-mediated off-target effects to the consistency of the two screens. We further investigated two seed-sequence properties, seed pairing stability, and target abundance in terms of their capability to minimize the off-target effects in post-screening data analysis. Finally, we applied this novel methodology to identify genetic interactions and synthetic lethal partners of cancer drivers, and confirmed differential essentiality phenotypes by detailed CRISPR/Cas9 experiments. Results: Using the novel concepts of seed essentiality and shRNA family, we demonstrate how genome-wide loss-of-function profiling of a common set of cancer cell lines can be actually made fairly reproducible when considering seed-mediated off-target effects. Importantly, by excluding shRNAs having higher propensity for off-target effects, based on their seed-sequence properties, one can remove noise from the genome-wide shRNA datasets. As a translational application case, we demonstrate enhanced reproducibility of genetic interaction partners of common cancer drivers, as well as identify novel synthetic lethal partners of a major oncogenic driver, PIK3CA, supported by a complementary CRISPR/Cas9 experiment. Conclusions: We provide practical guidelines for improved design and analysis of genome-wide loss-of-function profiling and demonstrate how this novel strategy can be applied towards improved mapping of genetic dependencies of cancer cells to aid development of targeted anticancer treatments.
  • Jaiswal, Alok; Peddinti, Gopal; Akimov, Yevhen; Wennerberg, Krister; Kuznetsov, Sergey; Tang, Jing; Aittokallio, Tero (BioMed Central, 2017)
    Abstract Background Genome-wide loss-of-function profiling is widely used for systematic identification of genetic dependencies in cancer cells; however, the poor reproducibility of RNA interference (RNAi) screens has been a major concern due to frequent off-target effects. Currently, a detailed understanding of the key factors contributing to the sub-optimal consistency is still a lacking, especially on how to improve the reliability of future RNAi screens by controlling for factors that determine their off-target propensity. Methods We performed a systematic, quantitative analysis of the consistency between two genome-wide shRNA screens conducted on a compendium of cancer cell lines, and also compared several gene summarization methods for inferring gene essentiality from shRNA level data. We then devised novel concepts of seed essentiality and shRNA family, based on seed region sequences of shRNAs, to study in-depth the contribution of seed-mediated off-target effects to the consistency of the two screens. We further investigated two seed-sequence properties, seed pairing stability, and target abundance in terms of their capability to minimize the off-target effects in post-screening data analysis. Finally, we applied this novel methodology to identify genetic interactions and synthetic lethal partners of cancer drivers, and confirmed differential essentiality phenotypes by detailed CRISPR/Cas9 experiments. Results Using the novel concepts of seed essentiality and shRNA family, we demonstrate how genome-wide loss-of-function profiling of a common set of cancer cell lines can be actually made fairly reproducible when considering seed-mediated off-target effects. Importantly, by excluding shRNAs having higher propensity for off-target effects, based on their seed-sequence properties, one can remove noise from the genome-wide shRNA datasets. As a translational application case, we demonstrate enhanced reproducibility of genetic interaction partners of common cancer drivers, as well as identify novel synthetic lethal partners of a major oncogenic driver, PIK3CA, supported by a complementary CRISPR/Cas9 experiment. Conclusions We provide practical guidelines for improved design and analysis of genome-wide loss-of-function profiling and demonstrate how this novel strategy can be applied towards improved mapping of genetic dependencies of cancer cells to aid development of targeted anticancer treatments.
  • Lioumis, Pantelis; Rosanova, Mario (2022)
    Transcranial magnetic stimulation combined with electroencephalography (TMS-EEG) allows measuring noninvasively the electrical response of the human cerebral cortex to a direct perturbation. Complementing TMSEEG with a structural neuronavigation tool (nTMS-EEG) is key for accurately selecting cortical areas, targeting them, and adjusting the stimulation parameters based on some relevant anatomical priors. This step, together with the employment of visualization tools designed to perform a quality check of TMS-evoked potentials (TEPs) in real-time during TMS-EEG data acquisition, is pivotal for maximizing the impact of the TMS pulse on the cortex and in ensuring highly reproducible measurements within sessions and across subjects. Moreover, storing stimulation parameters in the neumnavigation system can help in replicating the stimulation parameters within and across experimental sessions and sharing them across research centers. Finally, the systematic employment of neumnavigation in TMS-EEG studies is also critical to standardize measurements in clinical populations in search for reliable diagnostic and prognostic TMS-EEG-based biomarkers for neurological and psychiatric disorders.
  • Martins, João Pedro; das Neves, José; de la Fuente, María; Celia, Christian; Florindo, Helena; Günday-Türeli, Nazende; Popat, Amirali; Luis Santos, José; Sousa, Flávia; Schmid, Ruth; Wolfram, Joy; Sarmento, Bruno; Santos, Hélder A. (2020)
    This commentary article conveys the views of the board of the Nanomedicine and Nanoscale Delivery Focus Group of the Controlled Release Society regarding the decision of the United States National Cancer Institute (NCI) in halting funding for the Centers of Cancer Nanotechnology Excellence (CCNEs), and the subsequent editorial articles that broadened this discussion.
  • Rodionov, Andrey; Tolmacheva, Aleksandra; Kirveskari, Erika; Mäkelä, Jyrki P.; Shulga, Anastasia (2019)
    Background Accurate re-positioning of the coil is challenging in magnetic stimulation at the cervical spinal level. The applicability of coil location control for this type of stimulation is unexplored. New method Utilizing a figure-of-eight coil and anatomy-specific models of the magnetic stimulation system, we developed a novel technique that enables probing corticospinal excitability at the cervical spinal level. Magnetic stimulation was performed in 9 healthy subjects at C2-C6 spinal levels using a figure-of-eight coil and a coil tracking system. MEPs were recorded from the abductor digiti minimi muscle. The functioning of the coil tracking system was tested with an estimated electric field maximum (eEFM) above the C1 cervical level (group 1) and below (group 2). Motor-evoked potential (MEP) reproducibility was assessed with intra-class correlation coefficient (ICC). Results The use of coil location control in cervical level focal magnetic stimulation enabled the recording of highly reproducible MEPs. Within one co-registration, the ICC 95% confidence interval (CI) in group 1 was 0.89-0.99 and in group 2 was 0.24-0.85. Comparison with existing methods and conclusions This method can be used for accurate maintenance and retrieval of the focal coil position at the cervical level with low spatial variability during stimulation. Existing methodologies employ determination of the coil location based on external landmarks, which makes the procedure cumbersome. This technique can optimize existing stimulation protocols and facilitate development of navigated spinal stimulation.
  • Klump, Jens; Wyborn, Lesley; Wu, Mingfang; Martin, Julia; Downs, Robert R.; Asmi, Ari (2021)
    A dataset, small or big, is often changed to correct errors, apply new algorithms, or add new data (e.g., as part of a time series), etc. In addition, datasets might be bundled into collections, distributed in different encodings or mirrored onto different platforms. All these differences between versions of datasets need to be understood by researchers who want to cite the exact version of the dataset that was used to underpin their research. Failing to do so reduces the reproducibility of research results. Ambiguous identification of datasets also impacts researchers and data centres who are unable to gain recognition and credit for their contributions to the collection, creation, curation and publication of individual datasets. Although the means to identify datasets using persistent identifiers have been in place for more than a decade, systematic data versioning practices are currently not available. In this work, we analysed 39 use cases and current practices of data versioning across 33 organisations. We noticed that the term ‘version’ was used in a very general sense, extending beyond the more common understanding of ‘version’ to refer primarily to revisions and replacements. Using concepts developed in software versioning and the Functional Requirements for Bibliographic Records (FRBR) as a conceptual framework, we developed six foundational principles for versioning of datasets: Revision, Release, Granularity, Manifestation, Provenance and Citation. These six principles provide a high-level framework for guiding the consistent practice of data versioning and can also serve as guidance for data centres or data providers when setting up their own data revision and version protocols and procedures.