[DOCKTESTERS] [DOCKTERTERS] Sucessful validation for Consensus workflow for donors DO52621
Miguel Vazquez
mikisvaz at gmail.com
Mon May 29 08:01:07 EDT 2017
Hi all,
I've managed to validate the Consensus workflow for one donor, see result
below. I haven't managed to validate it for other donors yet because I'm
having trouble getting the files I need from GNOS, it doesn't seem to
connect to the tracker or something.
Comparison of for DO52621 using Consensus
---
Common: 8197
Extra: 0
Missing: 0
[Note that in the results above the counts are for pairs of
mutation-caller, not individual mutations, because it was easier to compare
this way.]
Best regards
Miguel
On Fri, May 26, 2017 at 10:29 AM, Miguel Vazquez <mikisvaz at gmail.com> wrote:
> Dear all,
>
> Following Solomons advice below I got the Consensus workflow to run and
> produce a result. I'll now see if I can validate the results, but at least
> its working.
>
> I had to add the following line (bold) to the Dockstore.json template
> (note that this undergoes some pattern replacement before its run, but you
> get the idea)
>
> "consensus_zipped_vcf": {
> "path": "[OUTPUT-DIR]/[DONOR].consensus.vcf.gz",
> "class": "File"
> },
> * "output_file_name":"[DONOR].consensus",*
> "consensus_vcf_index": {
> "path": "[OUTPUT-DIR]/[DONOR].consensus.vcf.gz.tbi",
> "class": "File"
> }
>
> Please update dockstore accordingly, this was missing from the
> automatically generated template
>
> Best regards
>
> Miguel
>
> On Tue, May 23, 2017 at 4:28 PM, Solomon Shorser <
> solomon.shorser at oicr.on.ca> wrote:
>
>> Hi Miguel,
>>
>> I am not sure about the COSMIC error. Did it prompt you for your username
>> and password? Are you able to access the files with sftp *outside* of a
>> docker-container context? I was not the author of this docker image or the
>> scripts within, but I can talk to the original author if we cannot solve
>> this ourselves.
>>
>> As for the CWLTool error:
>>
>> This error:
>>
>> *KeyError: 'output_file_name'*
>> *Workflow error, try again with --debug for more information:*
>> * 'output_file_name'*
>>
>> ...suggests that the variable named "output_file_name" was missing,
>> probably from the input json "workflow_params.json".
>> If your workflow_params.json file does not have an "output_file_name",
>> please try adding one (it should look like: ..."output_file_name":"myConsensus.vcf",...
>> in JSON) to the workflow_params.json file, and let me know how that works.
>>
>> Thanks,
>> Solomon Shorser.
>>
>>
>> On 2017-05-21 07:45 AM, Miguel Vazquez wrote:
>>
>> Dear all,
>>
>> I've ran into some more trouble with the Consensus workflow. In
>> particular its two issues that I'm not sure if they are related.
>>
>> First of all is that I followed the advice to download the data necessary
>> before hand:
>>
>> mkdir dbs
>> docker run -it -v "${PWD}/dbs":/dbs consensus_call download reference /dbs
>> docker run -it -v "${PWD}/dbs":/dbs consensus_call download annotations /dbs
>> docker run -it -v "${PWD}/dbs":/dbs consensus_call download cosmic /dbs
>>
>> However the last part that download COSMIC give me an error. I tried
>> using my login email address and password and I get 'Permission denied,
>> please try again.' I've check COSMIC several times and reset my password. I
>> have SFTP access enabled and everything. I'm not sure what is happening.
>>
>> Despite this I though that the workflow might be able to run, since the
>> COSMIC data is optional, but I get the following error
>>
>>
>> ubuntu at ip-10-42-6-176:~/DockerTest-Miguel$ cd
>> /mnt/1TB/work/DockerTest-Miguel/tests/Consensus/DO52621/ && dockstore
>> tool launch --script --entry quay.io/pancancer/pcawg-consen
>> sus-caller:latest --json Dockstore.json
>> Creating directories for run of Dockstore launcher at:
>> ./datastore//launcher-92578913-cbbf-4c57-9d19-35b83c4b6fb0
>> Provisioning your input files to your local machine
>> Calling out to cwltool to run your tool
>> Executing: cwltool --enable-dev --non-strict --outdir
>> /mnt/1TB/work/DockerTest-Miguel/tests/Consensus/DO52621/./
>> datastore/launcher-92578913-cbbf-4c57-9d19-35b83c4b6fb0/outputs/
>> --tmpdir-prefix /mnt/1TB/work/DockerTest-Migue
>> l/tests/Consensus/DO52621/./datastore/launcher-92578913-
>> cbbf-4c57-9d19-35b83c4b6fb0/tmp/ --tmp-outdir-prefix
>> /mnt/1TB/work/DockerTest-Miguel/tests/Consensus/DO52621/./
>> datastore/launcher-92578913-cbbf-4c57-9d19-35b83c4b6fb0/working/
>> /tmp/1495367250976-0/temp3097766058272663834.cwl
>> /mnt/1TB/work/DockerTest-Miguel/tests/Consensus/DO52621/./
>> datastore/launcher-92578913-cbbf-4c57-9d19-35b83c4b6fb0/
>> workflow_params.json
>> /usr/local/bin/cwltool 1.0.20161202203310
>> Resolved '/tmp/1495367250976-0/temp3097766058272663834.cwl' to '
>> file:///tmp/1495367250976-0/temp3097766058272663834.cwl'
>> Got workflow error
>> Traceback (most recent call last):
>> File "/usr/local/lib/python2.7/dist-packages/cwltool/main.py", line
>> 227, in single_job_executor
>> for r in jobiter:
>> File "/usr/local/lib/python2.7/dist-packages/cwltool/draft2tool.py",
>> line 370, in job
>> j.command_line = flatten(map(builder.generate_arg, builder.bindings))
>> File "/usr/local/lib/python2.7/dist-packages/cwltool/builder.py", line
>> 158, in generate_arg
>> value = self.do_eval(binding["valueFrom"], context=value)
>> File "/usr/local/lib/python2.7/dist-packages/cwltool/builder.py", line
>> 206, in do_eval
>> timeout=self.timeout)
>> File "/usr/local/lib/python2.7/dist-packages/cwltool/expression.py",
>> line 186, in do_eval
>> jslib=jslib)
>> File "/usr/local/lib/python2.7/dist-packages/cwltool/expression.py",
>> line 144, in interpolate
>> timeout=timeout)
>> File "/usr/local/lib/python2.7/dist-packages/cwltool/expression.py",
>> line 127, in evaluator
>> return next_seg(m.group(0)[m.end(1) - m.start(0):-1], obj[m.group(1)])
>> File "/usr/local/lib/python2.7/dist-packages/cwltool/expression.py",
>> line 113, in next_seg
>> return next_seg(remain[m.end(0):], obj[m.group(0)[1:]])
>> *KeyError: 'output_file_name'*
>> *Workflow error, try again with --debug for more information:*
>> * 'output_file_name'*
>> org.apache.commons.exec.ExecuteException: Process exited with an error:
>> 1 (Exit value: 1)
>> at org.apache.commons.exec.DefaultExecutor.executeInternal(Defa
>> ultExecutor.java:404)
>> at org.apache.commons.exec.DefaultExecutor.access$200(DefaultEx
>> ecutor.java:48)
>> at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecuto
>> r.java:200)
>> at java.lang.Thread.run(Thread.java:745)
>> java.lang.RuntimeException: problems running command: cwltool
>> --enable-dev --non-strict --outdir /mnt/1TB/work/DockerTest-Migue
>> l/tests/Consensus/DO52621/./datastore/launcher-92578913-
>> cbbf-4c57-9d19-35b83c4b6fb0/outputs/ --tmpdir-prefix
>> /mnt/1TB/work/DockerTest-Miguel/tests/Consensus/DO52621/./
>> datastore/launcher-92578913-cbbf-4c57-9d19-35b83c4b6fb0/tmp/
>> --tmp-outdir-prefix /mnt/1TB/work/DockerTest-Migue
>> l/tests/Consensus/DO52621/./datastore/launcher-92578913-
>> cbbf-4c57-9d19-35b83c4b6fb0/working/ /tmp/1495367250976-0/temp3097766058272663834.cwl
>> /mnt/1TB/work/DockerTest-Miguel/tests/Consensus/DO52621/./
>> datastore/launcher-92578913-cbbf-4c57-9d19-35b83c4b6fb0/
>> workflow_params.json
>>
>> Could this be related to the problem with COSMIC?
>>
>> Best regards
>>
>> Miguel
>>
>>
>> _______________________________________________
>> docktesters mailing listdocktesters at lists.icgc.orghttps://lists.icgc.org/mailman/listinfo/docktesters
>>
>>
>> *Solomon Shorser*
>> Software Developer
>>
>> *Ontario Institute for Cancer Research*
>> MaRS Centre
>> 661 University Avenue
>> Suite 510
>> Toronto, Ontario
>> Canada M5G 0A3
>>
>> Email: solomon.shorser at oicr.on.ca
>> Telephone: (647) 258-4319
>> Toll-free: 1-866-678-6427
>> Twitter: @OICR_news
>>
>> www.oicr.on.ca
>>
>> This message and any attachments may contain confidential and/or
>> privileged information for the sole use of the intended recipient. Any
>> review or distribution by anyone other than the person for whom it was
>> originally intended is strictly prohibited. If you have received this
>> message in error, please contact the sender and delete all copies.
>> Opinions, conclusions or other information contained in this message may
>> not be that of the organization.
>>
>> _______________________________________________
>> docktesters mailing list
>> docktesters at lists.icgc.org
>> https://lists.icgc.org/mailman/listinfo/docktesters
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.icgc.org/mailman/private/docktesters/attachments/20170529/590278aa/attachment-0001.html>
More information about the docktesters
mailing list