[DOCKTESTERS] Issues with Consensus Docker

Thu May 11 10:30:29 EDT 2017

Hi Miguel,

The .tbi files need to be there, but they do not need to be /explicitly/ 
included in the input JSON file.

CWL has the option to specify "secondary" files that go with a main 
file. In the CWL, an input VCF is defined like this example:

     dkfz_embl_input_file:
       type: File
       inputBinding:
         position: 3
         prefix: "-d"
_*      secondaryFiles:*__*
*__*        - .tbi*__*
*_

Specifying ".tbi" as secondary files indicates to CWL that it should 
look for any .tbi file that matches the same name as the actual file 
bound to this input should also be included (see: 
http://www.commonwl.org/v1.0/CommandLineTool.html#Expressions section 
5.2 CommandOutputParameter , "secondaryFiles").

As for the error related to creating the .tbi files: the format is 
similar to gzip but they should actually be created with a tool named 
"tabix" (I think you can install with apt on linux). Creating the tabix 
file can probably be done with a command that looks like this: "tabix -p 
vcf myDataFile.vcf.gz".

dbs_dir is supposed to be a Directory that contains the necessary 
references files. The original documentation from Jonathan Dursi shows 
there is a command to create this directory:

|mkdir dbs docker run -it -v "${PWD}/dbs":/dbs consensus_call download 
reference /dbs docker run -it -v "${PWD}/dbs":/dbs consensus_call 
download annotations /dbs docker run -it -v "${PWD}/dbs":/dbs 
consensus_call download cosmic /dbs|

(see: 
https://github.com/SolomonShorser-OICR/consensus_call_docker#downloading-auxiliary-files)

This is /not/ done within the dockstore tool as downloading this 
directory could take a long time (it is about 6 GB on my system) so I 
would recommend running the commands above before running the dockstore 
tool, then pointing the "dbs_dir" parameter to this directory.

I've attached a sample of the input JSON file I used.

Can you send me the link to the documentation you were reading? I think 
the dockstore usage needs to be made more clear, so please let me know 
where you were reading so I can update the right file.

Thanks,
Solomon Shorser.

On 2017-05-11 10:07 AM, Miguel Vazquez wrote:
> Hi all,
>
> I've started trying to test the Consensus Docker and I've ran into a 
> few difficulties. Some pertain the application itself and some I are 
> questions I have related to CWL or Docker.
>
> 1- Missing input dbs_dir: The documentation suggests that this is a 
> directory with resources that need to be set up beforehand using the 
> same docker container with different parameters. I'm not sure how to 
> handle this. To keep going I've just pointed to some arbitrary directory
>
> 2- Missing tbi files on input VCFs: The CWL explicitly asks for the 
> input VCF. In the documentation it seems like the inputs are in fact 
> compressed with gzip. Up to here is fine, though I wonder if there 
> would not be a more explicit way to indicate this in. The problem 
> comes when I run it that I get an error complaining about a missing 
> tbi file:
>
>   Missing input file [Errno 2] No such file or directory: 
> '/mnt/1TB/work/DockerTest-Miguel/data/DO52621//broad.oxoG.annotated.vcf.gz*.tbi*'
>
> It is not a problem to produce it using tabix but what is confusing to 
> me is that there is no mention of any tbi file in the Dockstore.json 
> file. How does this work?
>
> 3- After producing the tbi files I get this error, which I think is 
> related to (1)
>
>   Invalid filename: '' contains illegal characters
>
>
> Best regards
>
> Miguel
>
> ----Dockstore.json-----
>
>
> {
>   "broad_input_file": {
>     "path": "/tmp/fill_me_in.txt",
>     "class": "File"
>   },
>   "dkfz_embl_input_file": {
>     "path": "/tmp/fill_me_in.txt",
>     "class": "File"
>   },
>   "variant_type": "fill me in",
>   "consensus_vcf_index": {
>     "path": "/tmp/fill_me_in.txt",
>     "class": "File"
>   },
> *  "dbs_dir": {
>     "path": "/tmp/fill_directory_in",
>     "class": "Directory"
>   },*
>   "muse_input_file": {
>     "path": "/tmp/fill_me_in.txt",
>     "class": "File"
>   },
>   "consensus_zipped_vcf": {
>     "path": "/tmp/fill_me_in.txt",
>     "class": "File"
>   },
>   "sanger_input_file": {
>     "path": "/tmp/fill_me_in.txt",
>     "class": "File"
>   }
> }
>
>
>
>
> _______________________________________________
> docktesters mailing list
> docktesters at lists.icgc.org
> https://lists.icgc.org/mailman/listinfo/docktesters

*Solomon Shorser*
Software Developer

*Ontario Institute for Cancer Research*
MaRS Centre
661 University Avenue
Suite 510
Toronto, Ontario
Canada M5G 0A3

Email: solomon.shorser at oicr.on.ca
Telephone: (647) 258-4319
Toll-free: 1-866-678-6427
Twitter: @OICR_news

www.oicr.on.ca <http://www.oicr.on.ca/>

This message and any attachments may contain confidential and/or 
privileged information for the sole use of the intended recipient. Any 
review or distribution by anyone other than the person for whom it was 
originally intended is strictly prohibited. If you have received this 
message in error, please contact the sender and delete all copies. 
Opinions, conclusions or other information contained in this message may 
not be that of the organization.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.icgc.org/mailman/private/docktesters/attachments/20170511/8c80b723/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: consensus_calling_input0.json
Type: application/json
Size: 922 bytes
Desc: not available
URL: <https://lists.icgc.org/mailman/private/docktesters/attachments/20170511/8c80b723/attachment-0001.json>