[DOCKTESTERS] [DOCKTERTERS] Sucessful run for Consensus workflow. Validation pending
Miguel Vazquez
mikisvaz at gmail.com
Fri May 26 04:29:14 EDT 2017
Dear all,
Following Solomons advice below I got the Consensus workflow to run and
produce a result. I'll now see if I can validate the results, but at least
its working.
I had to add the following line (bold) to the Dockstore.json template (note
that this undergoes some pattern replacement before its run, but you get
the idea)
"consensus_zipped_vcf": {
"path": "[OUTPUT-DIR]/[DONOR].consensus.vcf.gz",
"class": "File"
},
* "output_file_name":"[DONOR].consensus",*
"consensus_vcf_index": {
"path": "[OUTPUT-DIR]/[DONOR].consensus.vcf.gz.tbi",
"class": "File"
}
Please update dockstore accordingly, this was missing from the
automatically generated template
Best regards
Miguel
On Tue, May 23, 2017 at 4:28 PM, Solomon Shorser <solomon.shorser at oicr.on.ca
> wrote:
> Hi Miguel,
>
> I am not sure about the COSMIC error. Did it prompt you for your username
> and password? Are you able to access the files with sftp *outside* of a
> docker-container context? I was not the author of this docker image or the
> scripts within, but I can talk to the original author if we cannot solve
> this ourselves.
>
> As for the CWLTool error:
>
> This error:
>
> *KeyError: 'output_file_name'*
> *Workflow error, try again with --debug for more information:*
> * 'output_file_name'*
>
> ...suggests that the variable named "output_file_name" was missing,
> probably from the input json "workflow_params.json".
> If your workflow_params.json file does not have an "output_file_name",
> please try adding one (it should look like: ..."output_file_name":"myConsensus.vcf",...
> in JSON) to the workflow_params.json file, and let me know how that works.
>
> Thanks,
> Solomon Shorser.
>
>
> On 2017-05-21 07:45 AM, Miguel Vazquez wrote:
>
> Dear all,
>
> I've ran into some more trouble with the Consensus workflow. In particular
> its two issues that I'm not sure if they are related.
>
> First of all is that I followed the advice to download the data necessary
> before hand:
>
> mkdir dbs
> docker run -it -v "${PWD}/dbs":/dbs consensus_call download reference /dbs
> docker run -it -v "${PWD}/dbs":/dbs consensus_call download annotations /dbs
> docker run -it -v "${PWD}/dbs":/dbs consensus_call download cosmic /dbs
>
> However the last part that download COSMIC give me an error. I tried using
> my login email address and password and I get 'Permission denied, please
> try again.' I've check COSMIC several times and reset my password. I have
> SFTP access enabled and everything. I'm not sure what is happening.
>
> Despite this I though that the workflow might be able to run, since the
> COSMIC data is optional, but I get the following error
>
>
> ubuntu at ip-10-42-6-176:~/DockerTest-Miguel$ cd /mnt/1TB/work/DockerTest-
> Miguel/tests/Consensus/DO52621/ && dockstore tool launch --script --entry
> quay.io/pancancer/pcawg-consensus-caller:latest --json Dockstore.json
> Creating directories for run of Dockstore launcher at:
> ./datastore//launcher-92578913-cbbf-4c57-9d19-35b83c4b6fb0
> Provisioning your input files to your local machine
> Calling out to cwltool to run your tool
> Executing: cwltool --enable-dev --non-strict --outdir
> /mnt/1TB/work/DockerTest-Miguel/tests/Consensus/
> DO52621/./datastore/launcher-92578913-cbbf-4c57-9d19-35b83c4b6fb0/outputs/
> --tmpdir-prefix /mnt/1TB/work/DockerTest-Miguel/tests/Consensus/
> DO52621/./datastore/launcher-92578913-cbbf-4c57-9d19-35b83c4b6fb0/tmp/
> --tmp-outdir-prefix /mnt/1TB/work/DockerTest-Miguel/tests/Consensus/
> DO52621/./datastore/launcher-92578913-cbbf-4c57-9d19-35b83c4b6fb0/working/
> /tmp/1495367250976-0/temp3097766058272663834.cwl /mnt/1TB/work/DockerTest-
> Miguel/tests/Consensus/DO52621/./datastore/launcher-
> 92578913-cbbf-4c57-9d19-35b83c4b6fb0/workflow_params.json
> /usr/local/bin/cwltool 1.0.20161202203310
> Resolved '/tmp/1495367250976-0/temp3097766058272663834.cwl' to '
> file:///tmp/1495367250976-0/temp3097766058272663834.cwl'
> Got workflow error
> Traceback (most recent call last):
> File "/usr/local/lib/python2.7/dist-packages/cwltool/main.py", line
> 227, in single_job_executor
> for r in jobiter:
> File "/usr/local/lib/python2.7/dist-packages/cwltool/draft2tool.py",
> line 370, in job
> j.command_line = flatten(map(builder.generate_arg, builder.bindings))
> File "/usr/local/lib/python2.7/dist-packages/cwltool/builder.py", line
> 158, in generate_arg
> value = self.do_eval(binding["valueFrom"], context=value)
> File "/usr/local/lib/python2.7/dist-packages/cwltool/builder.py", line
> 206, in do_eval
> timeout=self.timeout)
> File "/usr/local/lib/python2.7/dist-packages/cwltool/expression.py",
> line 186, in do_eval
> jslib=jslib)
> File "/usr/local/lib/python2.7/dist-packages/cwltool/expression.py",
> line 144, in interpolate
> timeout=timeout)
> File "/usr/local/lib/python2.7/dist-packages/cwltool/expression.py",
> line 127, in evaluator
> return next_seg(m.group(0)[m.end(1) - m.start(0):-1], obj[m.group(1)])
> File "/usr/local/lib/python2.7/dist-packages/cwltool/expression.py",
> line 113, in next_seg
> return next_seg(remain[m.end(0):], obj[m.group(0)[1:]])
> *KeyError: 'output_file_name'*
> *Workflow error, try again with --debug for more information:*
> * 'output_file_name'*
> org.apache.commons.exec.ExecuteException: Process exited with an error: 1
> (Exit value: 1)
> at org.apache.commons.exec.DefaultExecutor.executeInternal(
> DefaultExecutor.java:404)
> at org.apache.commons.exec.DefaultExecutor.access$200(
> DefaultExecutor.java:48)
> at org.apache.commons.exec.DefaultExecutor$1.run(
> DefaultExecutor.java:200)
> at java.lang.Thread.run(Thread.java:745)
> java.lang.RuntimeException: problems running command: cwltool --enable-dev
> --non-strict --outdir /mnt/1TB/work/DockerTest-Miguel/tests/Consensus/
> DO52621/./datastore/launcher-92578913-cbbf-4c57-9d19-35b83c4b6fb0/outputs/
> --tmpdir-prefix /mnt/1TB/work/DockerTest-Miguel/tests/Consensus/
> DO52621/./datastore/launcher-92578913-cbbf-4c57-9d19-35b83c4b6fb0/tmp/
> --tmp-outdir-prefix /mnt/1TB/work/DockerTest-Miguel/tests/Consensus/
> DO52621/./datastore/launcher-92578913-cbbf-4c57-9d19-35b83c4b6fb0/working/
> /tmp/1495367250976-0/temp3097766058272663834.cwl /mnt/1TB/work/DockerTest-
> Miguel/tests/Consensus/DO52621/./datastore/launcher-
> 92578913-cbbf-4c57-9d19-35b83c4b6fb0/workflow_params.json
>
> Could this be related to the problem with COSMIC?
>
> Best regards
>
> Miguel
>
>
> _______________________________________________
> docktesters mailing listdocktesters at lists.icgc.orghttps://lists.icgc.org/mailman/listinfo/docktesters
>
>
> *Solomon Shorser*
> Software Developer
>
> *Ontario Institute for Cancer Research*
> MaRS Centre
> 661 University Avenue
> Suite 510
> Toronto, Ontario
> Canada M5G 0A3
>
> Email: solomon.shorser at oicr.on.ca
> Telephone: (647) 258-4319
> Toll-free: 1-866-678-6427
> Twitter: @OICR_news
>
> www.oicr.on.ca
>
> This message and any attachments may contain confidential and/or
> privileged information for the sole use of the intended recipient. Any
> review or distribution by anyone other than the person for whom it was
> originally intended is strictly prohibited. If you have received this
> message in error, please contact the sender and delete all copies.
> Opinions, conclusions or other information contained in this message may
> not be that of the organization.
>
> _______________________________________________
> docktesters mailing list
> docktesters at lists.icgc.org
> https://lists.icgc.org/mailman/listinfo/docktesters
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.icgc.org/mailman/private/docktesters/attachments/20170526/de3542b9/attachment.html>
More information about the docktesters
mailing list