[DOCKTESTERS] Core workflow icgc reference data location and gosu
Denis Yuen
Denis.Yuen at oicr.on.ca
Mon Dec 5 12:47:40 EST 2016
Hi,
Yup, we've added that warning because we test with a specific version of cwltool.
It looks like you have a newer version of cwltool which is probably ok (assuming you haven't run into any problems). The warning is more aimed at users of older versions of cwltool than newer.
Denis Yuen
Bioinformatics Software Developer
Ontario Institute for Cancer Research
MaRS Centre
661 University Avenue
Suite 510
Toronto, Ontario, Canada M5G 0A3
Toll-free: 1-866-678-6427
Twitter: @OICR_news
www.oicr.on.ca<http://www.oicr.on.ca/>
This message and any attachments may contain confidential and/or privileged information for the sole use of the intended recipient. Any review or distribution by anyone other than the person for whom it was originally intended is strictly prohibited. If you have received this message in error, please contact the sender and delete all copies. Opinions, conclusions or other information contained in this message may not be that of the organization.
________________________________
From: mikisvaz at gmail.com [mikisvaz at gmail.com] on behalf of Miguel Vazquez [miguel.vazquez at cnio.es]
Sent: December 5, 2016 12:16 PM
To: Denis Yuen
Cc: Francis Ouellette; docktesters at lists.icgc.org
Subject: Re: [DOCKTESTERS] Core workflow icgc reference data location and gosu
By the way Denis, I now get this message when running dockstore:
cwltool version is 1.0.20161202203310 , Dockstore is tested with 1.0.20161114152756
Override and run with `--script`
I've added the --script to the command line and it works, but just so you know.
Miguel
On Mon, Dec 5, 2016 at 4:05 PM, Miguel Vazquez <miguel.vazquez at cnio.es<redir.aspx?REF=ecCYHu_4RoyycX0Hikg_on65eKkWeXhMLFUoglU1xi8SztJcNh3UCAFtYWlsdG86bWlndWVsLnZhenF1ZXpAY25pby5lcw..>> wrote:
Thanks Denis, your suggestion of removing the jar files worked.
Best regards
Miguel
On Mon, Dec 5, 2016 at 3:18 PM, Denis Yuen <Denis.Yuen at oicr.on.ca<redir.aspx?REF=LfYkayWO-wb3F2Jw63ufZJR00wdFYs8ZA_uWSSkRg6ISztJcNh3UCAFtYWlsdG86RGVuaXMuWXVlbkBvaWNyLm9uLmNh>> wrote:
Hi,
Having some difficulty reproducing this.
It's possible that a jar file was corrupted in transit, try deleting ~/.dockstore/self-installs/ and retrying.
________________________________
From: Francis Ouellette
Sent: December 3, 2016 6:48 PM
To: docktesters at lists.icgc.org<redir.aspx?REF=Qp7Y6coOhLdhUbtNUWXZO78T48bKG1p8vxPYuKWU9loSztJcNh3UCAFtYWlsdG86ZG9ja3Rlc3RlcnNAbGlzdHMuaWNnYy5vcmc.>
Cc: Denis Yuen; Miguel Vazquez
Subject: Re: [DOCKTESTERS] Core workflow icgc reference data location and gosu
Anybody else with these problems?
@bffo
On Dec 3, 2016, at 06:10, Miguel Vazquez <miguel.vazquez at cnio.es<redir.aspx?REF=3w7IBO4lxdWmG56Xs9HOk89mXNhNihEBiNbW5w2HlNNsMNVcNh3UCAFodHRwOi8vcmVkaXIuYXNweD9SRUY9Z0JwRTlBaUhtdWl1dkRVQUVuejN0VERvUXk5YXh5cjVjRW1FSlBJbEFfVm1LZHRkR1IzVUNBRnRZV2xzZEc4NmJXbG5kV1ZzTG5aaGVuRjFaWHBBWTI1cGJ5NWxjdy4u>> wrote:
Denis,
I've tried updating to the latest dockstore and ran into a bug.
ubuntu at ip-10-253-35-14:~$ dockstore
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: io/cwl/avro/CWL$GsonBuildException
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at sun.launcher.LauncherHelper.va<redir.aspx?REF=TjENSDOrNSff3TcU1xQl-epUGTBoxp9DLPeUa5dUwmZsMNVcNh3UCAFodHRwOi8vc3VuLmxhdW5jaGVyLkxhdW5jaGVySGVscGVyLnZh>lidateMainClass(LauncherHelper.java:544)
at sun.launcher.LauncherHelper.ch<redir.aspx?REF=2KqCm73z0xjg7eM3EbabyGtOB6GG-JE7vRwL8K3cDwJsMNVcNh3UCAFodHRwOi8vc3VuLmxhdW5jaGVyLkxhdW5jaGVySGVscGVyLmNo>eckAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: io.cwl.avro.CWL$GsonBuildException
at java.net.URLClassLoader$1.run(URLClassLoader.java:370)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 7 more
Caused by: java.util.zip.ZipException: invalid LOC header (bad signature)
at java.util.zip.ZipFile.read(Native Method)
at java.util.zip.ZipFile.access$1400(ZipFile.java:60)
at java.util.zip.ZipFile$ZipFileInputStream.read(ZipFile.java:717)
at java.util.zip.ZipFile$ZipFileInflaterInputStream.fill(ZipFile.java:419)
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158)
at sun.misc.Resource.getBytes(Resource.java:124)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:462)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
... 13 more
What I did is that I moved the old version of dockstore from ~/bin/dockstore to ~/bin/dockstore.old and downloaded the new version from
https://github.com/ga4gh/dockstore/releases/download/1.1/dockstore<redir.aspx?REF=sn9tvzEjp-Jjg6MLUJANNgI3ZFMSIVPNf4uGT8ytQolsMNVcNh3UCAFodHRwOi8vcmVkaXIuYXNweD9SRUY9dnR2Tmg3TTRtU002RW5YNVg1UkJRVDZ5dUMzNFAxRThBR1hzVUc3djVrM0FpOTFkR1IzVUNBRm9kSFJ3Y3pvdkwyZHBkR2gxWWk1amIyMHZaMkUwWjJndlpHOWphM04wYjNKbEwzSmxiR1ZoYzJWekwyUnZkMjVzYjJGa0x6RXVNUzlrYjJOcmMzUnZjbVUu>
I ran the new dockstore, it downloaded
https://seqwaremaven.oicr.on.ca/artifactory/collab-release/io/dockstore/dockstore-client/1.1/dockstore-client-1.1.jar<redir.aspx?REF=VtmsnE7YlRJ7qyRZN8OiEWFjV3un0ph7WIXVY0W3UQBsMNVcNh3UCAFodHRwOi8vcmVkaXIuYXNweD9SRUY9dmU4TjF0dUlGdDdROVBIeFVWYTdlN1hsRkxXQ0FRYzZrQmozdV9pb1l4VEFpOTFkR1IzVUNBRm9kSFJ3Y3pvdkwzTmxjWGRoY21WdFlYWmxiaTV2YVdOeUxtOXVMbU5oTDJGeWRHbG1ZV04wYjNKNUwyTnZiR3hoWWkxeVpXeGxZWE5sTDJsdkwyUnZZMnR6ZEc5eVpTOWtiMk5yYzNSdmNtVXRZMnhwWlc1MEx6RXVNUzlrYjJOcmMzUnZjbVV0WTJ4cFpXNTBMVEV1TVM1cVlYSS4.>
and then when I tried running dockstore --version I got the error above. Anything I can do?
Best regards
Miguel
On Fri, Dec 2, 2016 at 6:15 PM, Denis Yuen <Denis.Yuen at oicr.on.ca<redir.aspx?REF=wBWwq_FsU84z1f74zIwMTAqOFtqwCae0FREciFrpnXpsMNVcNh3UCAFodHRwOi8vcmVkaXIuYXNweD9SRUY9QjAybU5kWkxtQnphRm85VGMwejA5V3lMSU5ZWEcxeGQwd3JrTG1LLU5iSEFpOTFkR1IzVUNBRnRZV2xzZEc4NlJHVnVhWE11V1hWbGJrQnZhV055TG05dUxtTmg.>> wrote:
Hi,
I've gone through and created a new release for each of the core tools (bwa-mem, delly, dkfz, sanger) with two (non-scientific) changes.
1) Each of these tools has been redirected to use reference data hosted on the icgc portal ( https://dcc.icgc.org/releases/PCAWG<redir.aspx?REF=ojclhIiF6NoGskevsqPuwhFBSCaeGOgdCK97Zk6MRTJsMNVcNh3UCAFodHRwOi8vcmVkaXIuYXNweD9SRUY9eTkydDZmd09aUFdTeTFCeUZiVDE1SEJnYnZaRXlaVGFlMUw5Sjd5eVlQWEFpOTFkR1IzVUNBRm9kSFJ3Y3pvdkwyUmpZeTVwWTJkakxtOXlaeTl5Wld4bFlYTmxjeTlRUTBGWFJ3Li4.> ) rather than aws in order to conserve our use of S3. This means that sample json parameter files have been updated and any references inside the Docker images that we're aware of also have been updated. As usual, if re-running workflows frequently, you'll want to host these files locally.
If you are using the Dockstore command-line interface, you'll need to upgrade to the latest release of dockstore (1.2) to use this new file location.
2) A number of users have run into problems running the workflow when running in a multi-user environment (i.e. not running Docker containers with the first user on a host). This release replaces most usage of sudo inside the tools with gosu to deal with this issue.
The new release numbers are documented at https://wiki.oicr.on.ca/display/PANCANCER/Workflow+Testing+Data<redir.aspx?REF=AZHpukxYf-PZpiuBjB9AHNCNBoqNB3zBa1nX33XA3RlsMNVcNh3UCAFodHRwOi8vcmVkaXIuYXNweD9SRUY9Qk1EVW82cE9pVkdkZ0p2VExLVlpiQU5DT2U2UzFHTG9aNHlGbjlSbklmckFpOTFkR1IzVUNBRm9kSFJ3Y3pvdkwzZHBhMmt1YjJsamNpNXZiaTVqWVM5a2FYTndiR0Y1TDFCQlRrTkJUa05GVWk5WGIzSnJabXh2ZHl0VVpYTjBhVzVuSzBSaGRHRS4.>
In particular:
* bwa-mem 2.6.8_1.2
* dkfz 2.0.1_cwl1.0
* sanger 2.0.3
* embl 2.0.1-cwl1.0
Denis Yuen
Bioinformatics Software Developer
Ontario Institute for Cancer Research
MaRS Centre
661 University Avenue
Suite 510
Toronto, Ontario, Canada M5G 0A3
Toll-free: 1-866-678-6427
Twitter: @OICR_news
www.oicr.on.ca<redir.aspx?REF=oKirKLQEOi6F_dpAnFvVvGSA04rFj7YmZJzlaaMPWv5sMNVcNh3UCAFodHRwOi8vcmVkaXIuYXNweD9SRUY9NkVOeEFvRlY2OU1wcGlCOFVac1c3UzYyLTFHTDBtazlHNldXdGZWQjJvM0FpOTFkR1IzVUNBRm9kSFJ3T2k4dmQzZDNMbTlwWTNJdWIyNHVZMkV2>
This message and any attachments may contain confidential and/or privileged information for the sole use of the intended recipient. Any review or distribution by anyone other than the person for whom it was originally intended is strictly prohibited. If you have received this message in error, please contact the sender and delete all copies. Opinions, conclusions or other information contained in this message may not be that of the organization.
_______________________________________________
docktesters mailing list
docktesters at lists.icgc.org<redir.aspx?REF=V7WAXgfDDWMtWRn-104J6lG8D5XIMOdSNSLaa741nwBsMNVcNh3UCAFodHRwOi8vcmVkaXIuYXNweD9SRUY9VmN4Yk03bEF5bXZkM19GUy15SDQxaWtON3VYM2F4VEVaRFY2Uk1WV1FWSEFpOTFkR1IzVUNBRnRZV2xzZEc4NlpHOWphM1JsYzNSbGNuTkFiR2x6ZEhNdWFXTm5ZeTV2Y21jLg..>
https://lists.icgc.org/mailman/listinfo/docktesters<redir.aspx?REF=MV-kfX0Z-zUyJ03K-X9-W_0RtXZD33IWdJoc7399uP5sMNVcNh3UCAFodHRwOi8vcmVkaXIuYXNweD9SRUY9RFlWRkl6c2szeHhjUHp0YVUtbXljQTg5Mm9XYnJMVHp6d0pUTVREUW9BTEFpOTFkR1IzVUNBRm9kSFJ3Y3pvdkwyeHBjM1J6TG1saloyTXViM0puTDIxaGFXeHRZVzR2YkdsemRHbHVabTh2Wkc5amEzUmxjM1JsY25NLg..>
_______________________________________________
docktesters mailing list
docktesters at lists.icgc.org<redir.aspx?REF=V7WAXgfDDWMtWRn-104J6lG8D5XIMOdSNSLaa741nwBsMNVcNh3UCAFodHRwOi8vcmVkaXIuYXNweD9SRUY9VmN4Yk03bEF5bXZkM19GUy15SDQxaWtON3VYM2F4VEVaRFY2Uk1WV1FWSEFpOTFkR1IzVUNBRnRZV2xzZEc4NlpHOWphM1JsYzNSbGNuTkFiR2x6ZEhNdWFXTm5ZeTV2Y21jLg..>
https://lists.icgc.org/mailman/listinfo/docktesters<redir.aspx?REF=i95dEK5NpCAmHm-E-jSFaGbp3L4f04hqFETUxxBCXWZsMNVcNh3UCAFodHRwczovL2xpc3RzLmljZ2Mub3JnL21haWxtYW4vbGlzdGluZm8vZG9ja3Rlc3RlcnM.>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.icgc.org/mailman/private/docktesters/attachments/20161205/aa4c4c06/attachment-0001.html>
More information about the docktesters
mailing list