{"id":51,"date":"2018-02-13T13:19:14","date_gmt":"2018-02-13T13:19:14","guid":{"rendered":"https:\/\/blogs-staging.imperial.ac.uk\/research-software-engineering\/?p=51"},"modified":"2019-06-11T12:39:23","modified_gmt":"2019-06-11T12:39:23","slug":"cloud-first-simple-automated-testing-using-drone","status":"publish","type":"post","link":"https:\/\/blogs-staging.imperial.ac.uk\/research-software-engineering\/2018\/02\/13\/cloud-first-simple-automated-testing-using-drone\/","title":{"rendered":"Cloud-first: Simple automated testing using Drone"},"content":{"rendered":"<p><em>This is the first in a <a href=\"https:\/\/blogs-staging.imperial.ac.uk\/research-software-engineering\/tag\/rse-cloud-computing-award\/\">series of posts<\/a> describing activities funded by our <a href=\"https:\/\/www.software.ac.uk\/rse-cloud-awardees\">RSE Cloud Computing Award<\/a>. We are exploring the use of selected <a href=\"https:\/\/azure.microsoft.com\/en-gb\/\">Microsoft Azure<\/a> services to accelerate the delivery of RSE projects via a cloud-first approach.<\/em><\/p>\n<p>A great way to explore an unfamiliar cloud platform is to deploy a familiar tool and compare the process with that used for an on-premise installation. In this case we&#8217;ll set up an open source continuous delivery system (<a href=\"https:\/\/drone.io\/\">Drone<\/a>) to carry out automated testing of a simple Python project hosted on GitHub. Drone is not as capable or flexible as alternatives like\u00a0<a href=\"https:\/\/jenkins.io\/\">Jenkins<\/a> (which we&#8217;ll consider in a subsequent post) but it&#8217;s a lot simpler and a suitable example of a self-contained webapp for our purposes of getting started with Azure.<\/p>\n<p>We&#8217;ll be automatically testing\u00a0<a href=\"https:\/\/github.com\/mwoodbri\/drone-example\/tree\/2d838c52c671b85890c619b11244aa3557e62146\">this repository<\/a>, containing a trivial Python 3 project with a single test which can be run via <span style=\"font-family: 'courier new', courier, monospace\">python -m unittest<\/span>.\u00a0 We add a <a href=\"https:\/\/github.com\/mwoodbri\/drone-example\/blob\/master\/.drone.yml\">single YAML file<\/a> to the repository to configure Drone accordingly.<\/p>\n<p>There are then just three (short!) steps to get Drone testing the repository whenever code is pushed to GitHub. You don&#8217;t need anything except a web browser and an Azure account:<\/p>\n<p><strong>1: Create an Azure VM where we&#8217;ll install Drone<\/strong><\/p>\n<p>You can do this via the <a href=\"https:\/\/portal.azure.com\/\">Azure Portal<\/a>\u00a0but we&#8217;ll use the new\u00a0<a href=\"https:\/\/shell.azure.com\/\">Azure Cloud Shell<\/a>\u00a0as it&#8217;s quicker &#8211; and easier to document, which is important for reproducibility. Drone is distributed as a Docker image so we&#8217;ll provision a minimal\u00a0<a href=\"https:\/\/coreos.com\/os\/docs\/latest\/\">Container Linux<\/a>\u00a0VM to host it. We need to create a resource group, add the VM, give it a public DNS name (you will need to choose your own, instead of <span style=\"font-family: 'courier new', courier, monospace\">my-ci-server<\/span>) and enable HTTP(S) access:<\/p>\n<pre>az group create -l westeurope --name my-rg<\/pre>\n<pre>az vm create --name my-ci-server --resource-group my-rg --image CoreOS:CoreOS:Stable:1632.2.1 --generate-ssh-keys --size Basic_A0<\/pre>\n<pre>az network public-ip update --name my-ci-serverPublicIP --resource-group my-rg --dns-name <strong>my-ci-server<\/strong><\/pre>\n<pre>az network nsg rule create --resource-group my-rg --nsg-name my-ci-serverNSG --name HTTP --destination-port-ranges 80 --priority 1010<\/pre>\n<pre>az network nsg rule create --resource-group my-rg --nsg-name my-ci-serverNSG --name HTTPS --destination-port-ranges 443 --priority 1020<\/pre>\n<p><strong>2: Register a new OAuth application in GitHub<\/strong><\/p>\n<p>In order to provide Drone with access to the repository (or repositories) we want to test, visit <a href=\"https:\/\/github.com\/settings\/applications\/new\">this page<\/a> and enter the following, replacing the hostname appropriately:<\/p>\n<ul>\n<li>Application name: Drone<\/li>\n<li>Homepage URL: <span style=\"font-family: 'courier new', courier, monospace\">https:\/\/<strong>my-ci-server<\/strong>.westeurope.cloudapp.azure.com<\/span><\/li>\n<li>Authorization callback URL: <span style=\"font-family: 'courier new', courier, monospace\">https:\/\/<strong>my-ci-server<\/strong>.westeurope.cloudapp.azure.com\/authorize<\/span><\/li>\n<\/ul>\n<p>Save the\u00a0Client ID and Client Secret for the next step<\/p>\n<p><strong>3: Install and configure Drone<\/strong><\/p>\n<p>Run the following commands back in the Cloud Shell. You again need to replace the hostname, and also provide your GitHub username and the Client ID and Secret from the previous step.<\/p>\n<pre>ssh <strong>my-ci-server<\/strong>.westeurope.cloudapp.azure.com<\/pre>\n<pre>sudo docker run -d --name drone-server -e DRONE_HOST=https:\/\/<strong>my-ci-server<\/strong>.westeurope.cloudapp.azure.com -e DRONE_ADMIN=<strong>mwoodbri<\/strong> -e DRONE_GITHUB=true -e DRONE_GITHUB_CLIENT=<strong>xxxxxxxxxxxxxxxxxxxx<\/strong> -e DRONE_GITHUB_SECRET=<strong>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx<\/strong> -e DRONE_LETS_ENCRYPT=true -v drone:\/var\/lib\/drone\/ -p 80:80 -p 443:443 --restart=unless-stopped drone\/drone<\/pre>\n<pre>sudo docker run -d --name drone-agent --link drone-server -e DRONE_SERVER=drone-server:9000 -v \/var\/run\/docker.sock:\/var\/run\/docker.sock --restart=unless-stopped drone\/agent<\/pre>\n<p>Then visit\u00a0<span style=\"font-family: 'courier new', courier, monospace\">https:\/\/<strong>my-ci-server<\/strong>.westeurope.cloudapp.azure.com<\/span> and toggle the switch next to the name of the relevant repository.<\/p>\n<p><b>Next steps<\/b><\/p>\n<p>Drone is now monitoring the code for changes, and will run the test suite in response. If we deliberately break our unit test by making <a href=\"https:\/\/github.com\/mwoodbri\/drone-example\/commit\/1af9ccaecaf570945b8059702c351dd829503b68\">this change<\/a>\u00a0and pushing the code then Drone will immediately run the code and identify a problem:<\/p>\n<p><a href=\"https:\/\/blogs-staging.imperial.ac.uk\/research-software-engineering\/files\/2018\/02\/Screen-Shot-2018-02-11-at-14.58.14.png\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"433\" class=\"alignnone wp-image-58 size-large\" src=\"https:\/\/blogs-staging.imperial.ac.uk\/research-software-engineering\/files\/2018\/02\/Screen-Shot-2018-02-11-at-14.58.14-1024x433.png\" alt=\"\" \/><\/a><\/p>\n<p>It will also annotate <a href=\"https:\/\/github.com\/mwoodbri\/drone-example\/commits\/master\">the commit<\/a> as bad and provide us with a badge that can be dynamically embedded in our <a href=\"https:\/\/github.com\/mwoodbri\/drone-example\/blob\/master\/README.md\">README.md<\/a>.<\/p>\n<p>We can then go onto configure Drone to notify us via email, Slack etc of failures using one of its <a href=\"http:\/\/plugins.drone.io\/\">many plugins<\/a>.<\/p>\n<p><strong>Summary<\/strong><\/p>\n<p>We&#8217;ve seen how various features of the Azure platform, including Virtual Machines, Cloud Shell, and the extensive Marketplace can be combined with GitHub and Drone to rapidly deploy a secure, private CI system entirely from your browser. There exist alternative means of achieving the same result &#8211; not least various hosted, subscription based systems &#8211; and there are Azure recipes for <a href=\"https:\/\/docs.microsoft.com\/en-gb\/azure\/jenkins\/install-jenkins-solution-template\">Jenkins<\/a> and <a href=\"https:\/\/azuremarketplace.microsoft.com\/en-gb\/marketplace\/apps\/drone.drone\">Drone<\/a> itself. However, the approach demonstrated here is applicable to any container-based software and therefore provides a flexible and efficient means of at least prototyping new services &#8211; via a cloud-first strategy.<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This is the first in a series of posts describing activities funded by our RSE Cloud Computing Award. We are exploring the use of selected Microsoft Azure services to accelerate the delivery of RSE projects via a cloud-first approach. A great way to explore an unfamiliar cloud platform is to deploy a familiar tool and [&hellip;]<\/p>\n","protected":false},"author":1133,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[390],"tags":[289397,289169],"class_list":["post-51","post","type-post","status-publish","format-standard","hentry","category-technology","tag-azure","tag-rse-cloud-computing-award"],"_links":{"self":[{"href":"https:\/\/blogs-staging.imperial.ac.uk\/research-software-engineering\/wp-json\/wp\/v2\/posts\/51","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs-staging.imperial.ac.uk\/research-software-engineering\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs-staging.imperial.ac.uk\/research-software-engineering\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs-staging.imperial.ac.uk\/research-software-engineering\/wp-json\/wp\/v2\/users\/1133"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs-staging.imperial.ac.uk\/research-software-engineering\/wp-json\/wp\/v2\/comments?post=51"}],"version-history":[{"count":15,"href":"https:\/\/blogs-staging.imperial.ac.uk\/research-software-engineering\/wp-json\/wp\/v2\/posts\/51\/revisions"}],"predecessor-version":[{"id":70,"href":"https:\/\/blogs-staging.imperial.ac.uk\/research-software-engineering\/wp-json\/wp\/v2\/posts\/51\/revisions\/70"}],"wp:attachment":[{"href":"https:\/\/blogs-staging.imperial.ac.uk\/research-software-engineering\/wp-json\/wp\/v2\/media?parent=51"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs-staging.imperial.ac.uk\/research-software-engineering\/wp-json\/wp\/v2\/categories?post=51"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs-staging.imperial.ac.uk\/research-software-engineering\/wp-json\/wp\/v2\/tags?post=51"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}