[DOCKTESTERS] Variant call validation results for Sanger

Tue Dec 6 10:52:52 EST 2016

Hi Alex,

Likewise here, on the test that “worked”, are the differences platform specific, and
are they reproducible.

I think we only need to do this a couple of times, to inform us if the differences are
operator  and/or platform specific, or simply (which I think) more about the heuristics
of the testing we are doing.

Thank you for looking into this.

@bffo

--
B.F. Francis Ouellette          http://oicr.on.ca/person/francis-ouellette

On Dec 5, 2016, at 1:21 PM, Alexander Buchanan <buchanae at ohsu.edu<mailto:buchanae at ohsu.edu>> wrote:

Regarding the validation results I posted last Friday, we think these poor results are likely due an upstream issue and not the Sanger workflow itself. Those variant call results were from a larger process including fastq prep., alignment, and then Sanger variant calling, and we think we introduced a problem early on during fastq prep.

We have a different set of Sanger results that reused the existing alignments from GNOS, and those variants match the expected results much more closely (99.99% match).  One example output from USeq:

82486    Key variants
82486    Key variants in shared regions
0.953626071716167    Shared key variants Ti/Tv
82482    Test variants
82482    Test variants in shared regions
0.9536238749407864    Shared test variants Ti/Tv

QUALThreshold    NumMatchTest    NumNonMatchTest    FDR=nonMatchTest/(matchTest+nonMatchTest)    decreasingFDR    TPR=matchTest/totalKeyFPR=nonMatchTest/totalKey    PPV=matchTest/(matchTest+nonMatchTest)
none    82479    3    3.6371574E-5    3.6371574E-5    0.9999151    3.636981E-5    0.99996364

From: <docktesters-bounces+buchanae=ohsu.edu at lists.icgc.org<mailto:docktesters-bounces+buchanae=ohsu.edu at lists.icgc.org>> on behalf of Alexander Buchanan <buchanae at ohsu.edu<mailto:buchanae at ohsu.edu>>
Date: Friday, December 2, 2016 at 4:11 PM
To: "docktesters at lists.icgc.org<mailto:docktesters at lists.icgc.org>" <docktesters at lists.icgc.org<mailto:docktesters at lists.icgc.org>>
Subject: [DOCKTESTERS] Variant call validation results for Sanger

I was able to run USeq on data output from running the sanger workflow on a Cromwell engine, for 5 donors. It’s reporting some pretty big differences, so I still need investigate. I’ll copy the USeq output at the end of this email.

I also wrote a simple comparison script, similar to what Miguel is doing (but in python), and it also reports differences.

At this point, I don’t know the source of the difference. Maybe I’m not comparing the data correctly, or maybe the workflows were run incorrectly. Maybe the tools have some element of randomness, but I’m not sure that would explain the substantial differences.

Output from python:

python test.py
==================================================
Donor:  DO50414
intersection 17395
key - test 10904
test - key 6722
==================================================
Donor:  DO50415
intersection 34721
key - test 17806
test - key 8755
==================================================
Donor:  DO50417
intersection 81477
key - test 39521
test - key 15959
==================================================
Donor:  DO50419
intersection 82705
key - test 41674
test - key 15262
==================================================
Donor:  DO50432
intersection 3941
key - test 24358
test - key 138224

Output from USeq:

[2 Dec 2016 15:09] USeq_8.9.6 Arguments: -a key-data/DO50414/somatic_snv_mnv.vcf.gz -b human_chromosomes.bed -c ccc-output/DO50414/somatic_snv_mnv.vcf.gz -d human_chromosomes.bed

VCF Comparator Settings:

somatic_snv_mnv.vcf.gz  Key vcf file
human_chromosomes.bed              Key interrogated regions file
somatic_snv_mnv.vcf.gz  Test vcf file
human_chromosomes.bed              Test interrogated regions file
true        Require matching alternate bases
false       Require matching genotypes
false       Use record VQSLOD score as ranking statistic
false       Exclude non PASS or . records
true        Compare all variant