{"id":317,"date":"2020-02-16T17:07:02","date_gmt":"2020-02-16T17:07:02","guid":{"rendered":"https:\/\/blogs-staging.imperial.ac.uk\/research-software-engineering\/?p=317"},"modified":"2020-02-17T08:27:54","modified_gmt":"2020-02-17T08:27:54","slug":"running-jupyter-notebooks-on-imperial-colleges-compute-cluster","status":"publish","type":"post","link":"https:\/\/blogs-staging.imperial.ac.uk\/research-software-engineering\/2020\/02\/16\/running-jupyter-notebooks-on-imperial-colleges-compute-cluster\/","title":{"rendered":"Running Jupyter notebooks on Imperial College&#8217;s compute cluster"},"content":{"rendered":"<p>We were really glad to see <a href=\"http:\/\/www.jamesphoward.com\/\">James Howard<\/a> (NHLI, Faculty of Medicine) <a href=\"https:\/\/twitter.com\/DrJHoward\/status\/1218108204833525760\">announcing on Twitter<\/a> that he&#8217;d published a <a href=\"https:\/\/www.kaggle.com\/jamesphoward\/pacemaker-identification-neural-network-example\">Kaggle kernel<\/a> to accompany his <a href=\"http:\/\/electrophysiology.onlinejacc.org\/content\/5\/5\/576\">recent publication<\/a> on MR image analysis for cardiac pacemaker identification using neural networks via PyTorch and torchvision. Sharing code in this way is a great way to promote <a href=\"https:\/\/www.imperial.ac.uk\/research-and-innovation\/support-for-staff\/scholarly-communication\/imperial-open-research-newsletter\/\">open research<\/a>, enable reproducibility and encourage re-use.<\/p>\n<figure id=\"attachment_323\" aria-describedby=\"caption-attachment-323\" style=\"width: 640px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"1013\" height=\"1024\" class=\"size-large wp-image-323\" src=\"https:\/\/blogs-staging.imperial.ac.uk\/research-software-engineering\/files\/2020\/02\/F5.large_-1013x1024.jpg\" alt=\"\" data-wp-editing=\"1\" \/><figcaption id=\"caption-attachment-323\" class=\"wp-caption-text\">Figure 3 from <a href=\"http:\/\/electrophysiology.onlinejacc.org\/content\/5\/5\/576\" target=\"_blank\" rel=\"noopener noreferrer\">Cardiac Rhythm Device Identification Using Neural Networks<\/a><\/figcaption><\/figure>\n<p>We thought it might be helpful to explain how to run similar notebooks on Imperial&#8217;s <a href=\"https:\/\/www.imperial.ac.uk\/admin-services\/ict\/self-service\/research-support\/rcs\/computing\/\">cluster compute service<\/a>, given that it can provide some benefits while you&#8217;re developing code:<\/p>\n<ul>\n<li>Your code and data remain securely on-premise, thanks to the RCS <a href=\"https:\/\/www.imperial.ac.uk\/admin-services\/ict\/self-service\/research-support\/rcs\/support\/applications\/jupyter\/\">Jupyter Service<\/a> and <a href=\"http:\/\/www.imperial.ac.uk\/admin-services\/ict\/self-service\/research-support\/rcs\/rds\/\">Research Data Store<\/a><\/li>\n<li>You can run parallel interactive and non-interactive jobs that span several days, across multiple <a href=\"https:\/\/www.imperial.ac.uk\/admin-services\/ict\/self-service\/research-support\/rcs\/computing\/job-sizing-guidance\/gpu\/\">GPUs<\/a><\/li>\n<\/ul>\n<p>With James&#8217; permission we&#8217;ve lightly modified his notebook and published it in an <a href=\"https:\/\/github.com\/ImperialCollegeLondon\/rcs-pacemakers\">exemplar repository<\/a> alongside some instructions to run it on the compute cluster. We hope this can help others to use a combination of Conda, Jupyter and PBS in order to conduct GPU-accelerated machine learning on infrastructure managed by the College&#8217;s <a href=\"https:\/\/www.imperial.ac.uk\/admin-services\/ict\/self-service\/research-support\/rcs\/computing\/\">Research Computing Service<\/a> &#8211; without incurring any cost at the point of use.<\/p>\n<p><em>Many thanks to James Howard for sharing his notebook and reviewing our instructions<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>We were really glad to see James Howard (NHLI, Faculty of Medicine) announcing on Twitter that he&#8217;d published a Kaggle kernel to accompany his recent publication on MR image analysis for cardiac pacemaker identification using neural networks via PyTorch and torchvision. Sharing code in this way is a great way to promote open research, enable [&hellip;]<\/p>\n","protected":false},"author":1133,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[390],"tags":[289406,289405,272455],"class_list":["post-317","post","type-post","status-publish","format-standard","hentry","category-technology","tag-gpu","tag-jupyter","tag-machine-learning"],"_links":{"self":[{"href":"https:\/\/blogs-staging.imperial.ac.uk\/research-software-engineering\/wp-json\/wp\/v2\/posts\/317","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs-staging.imperial.ac.uk\/research-software-engineering\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs-staging.imperial.ac.uk\/research-software-engineering\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs-staging.imperial.ac.uk\/research-software-engineering\/wp-json\/wp\/v2\/users\/1133"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs-staging.imperial.ac.uk\/research-software-engineering\/wp-json\/wp\/v2\/comments?post=317"}],"version-history":[{"count":11,"href":"https:\/\/blogs-staging.imperial.ac.uk\/research-software-engineering\/wp-json\/wp\/v2\/posts\/317\/revisions"}],"predecessor-version":[{"id":329,"href":"https:\/\/blogs-staging.imperial.ac.uk\/research-software-engineering\/wp-json\/wp\/v2\/posts\/317\/revisions\/329"}],"wp:attachment":[{"href":"https:\/\/blogs-staging.imperial.ac.uk\/research-software-engineering\/wp-json\/wp\/v2\/media?parent=317"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs-staging.imperial.ac.uk\/research-software-engineering\/wp-json\/wp\/v2\/categories?post=317"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs-staging.imperial.ac.uk\/research-software-engineering\/wp-json\/wp\/v2\/tags?post=317"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}