Institute for Security Science & Technology

Cyber attacks are considered one of the major threats for national security by the UK government. Artificial intelligence is considered to be a technology with major potential benefit. But what happens when these two worlds combine?

That’s exactly what the House of Lords Select Committee on Artificial Intelligence wanted to know. To find out more, they recently called in Professor Chris Hankin, Co-Director of the Institute for Security Science and Technology, to provide the panel with professional insight.

Below is a cut and edited summary of the evidence session. Some of the questions included have been rephrased. You can watch the full session online here.

What does artificial intelligence mean for cyber security today?

When I think about artificial intelligence in the context of cyber security today, I think mainly about machine learning, rather than broad artificial intelligence.

At Imperial, researchers have had success in using machine learning to analyse network traffic, learn what “normal” looks like, and spot anomalous things which might be indicative of a cyber attack.

This sort of approach is also used, for example, by the Darktrace, a UK company.

How successful is this approach?

It is a very exciting technology, and Darktrace has made a great commercial success out of it.

There are still some open research challenges to giving more accurate signals about what is going on, and reducing false positives. This is the focus of academic research across the world.

What might future developments of AI in cyber security look like?

In August 2016, a competition was held in the United States to develop automatic defensive systems that could understand when they were under attack, and then repair themselves and mitigate against the attack. Over, say, a 10 to 15-year horizon, we could be looking at that sort of technology being lifted to the level of systems. People often use the analogy of the human immune system when describing this potential technology.

Will only state-sponsored hackers have the means to deploy AI in cyber attacks, or is there a risk that AI-enabled cyber attacks will be “democratised” in the near future?

As Dr. Mark Briers articulated during his answer in the House of Lords, many of the “democratised” threats we see today probably came from state sponsored efforts some 10 years ago. Earlier this year in fact we saw hacking tools that were developed by the NSA being leaked online by a criminal hacking group. Looking forward 10 years, we might expect AI cyber weapons to follow the same path, from initially being developed by states, to becoming widely available.

This creates an additional problem in attribution; it is becoming much more difficult to differentiate between state actors and organised crime, as the sorts of techniques that those two groups are using to mount cyber attacks are increasingly similar.

Adversarial AI, which aims to disrupt artificial intelligence learning systems, is a current research topic. How much of an issue are recent developments in that field of adversarial AI for the deployment of AI systems in cyber security?

We have been doing some work on using adversarial AI to see how possible it is to train an attacker to evade the state-of-the-art cyber security detection algorithms, called classifiers, of the type we discussed earlier.

We’ve seen that if you can get into the right part of the system, you can learn a lot about what the cyber security classifier might be doing, and introduce noise into your attack to evade detection. The message I take from this is that, at the moment, AI is not the only answer we should be thinking about for defending our systems.

For example, let’s think about the Stuxnet malware that was used to delay the Iranians in their uranium enrichment process. The attack was essentially a physical attack, mounted through cyber, and in one version at least it caused the rotor blades in the enrichment centrifuges to spin at very high speeds.

An AI detector might have been able to detect that attack by looking at some network traffic, or maybe the adversarial AI approach might have evaded detection. Either way, if you had been standing anywhere near the centrifuges you would also have had a physical signal that something was going wrong.

How prepared is the UK for the impact of artificial intelligence on cyber security?

The UK’s NCSC has produced some very good advice for companies, government and private citizens about how to protect themselves. The sorts of attacks that we may be talking about, which are AI-based, will at the moment be probably no different from the sorts of attacks you see from human attackers, and so this advice is still valid.

Advice around cyber hygiene, such as keeping software up to date, having appropriate antivirus software, not sharing passwords with people etc. is very effective in reducing the impact of cyber threats. Unfortunately, the cyber attacks that have been most prominent in the news over the last year—WannaCry, NotPetya, Equifax—have all been the consequence of people running unpatched software, contrary to this advice.

What, in your view, is the single most important policy recommendation?

For the future, it is very important that we recognise that cyber security is a priority within the artificial intelligence area, and that a good number of studentships at all levels are funded to support this linkage between cyber security and AI.

Read What does artificial intelligence mean for cyber security? Prof Chris Hankin speaks to the House of Lords Select Committee. in full

A post by Professor Emil Lupu, Associate Director of the ISST and Director of the Academic Centre of Excellence in Cyber Security Research.

It’s often reported that we can expect 30 billion IoT devices in the world by 2020, creating webs of cyber-physical systems that combine the digital, physical and human dimensions.

In the not too distant future, an autonomous car will zip you through the ‘smart’ city, conversing with the nearby vehicles and infrastructure to adapt its route and speed. As you sit in the back seat, tiny medical devices might measure your vitals and send updates to your doctor for your upcoming appointment. All of this will rely on IoT devices; internet-connected sensors and actuators dispersed throughout our physical environment, even inside our bodies.

On the minds of many, but not so often reported, is that by bringing the digital interface into the system you make it reachable from anywhere on the internet, and therefore, also to malicious actors. And by taking the computer out of a secure room, and putting it for example at street level, you make it vulnerable to someone physically compromising it. Can we trust these cyber-physical systems?

Sensors can lie

So what might these malicious actors do? At Imperial College London, we’ve shown that sensors which, for example, monitor for fires, volcano eruptions and health signals, can be made to lie about the data they report. This can have drastic consequences.

The below charts show bedside-sensor data from a healthcare setting. On the first chart, each vertical, dotted line represents an event when the health of the patient has been at risk. By compromising three sensors, as shown on the second chart, we can cancel all of these points and mask the events.

The consequences of this happening in the real world could be fatal. So we have started working on techniques to detect when sensors might be lying, by measuring the correlations between the measurements of different sensors.

Catching a lie

Using our techniques with fire sensors, we could detect a fake fire event even when it was located next to a genuine fire event. The below charts show the fire detection system – the chart on the right clearly highlights the fake event.

This also allows us to detect masked events – when someone is trying to hide an intrusion – and is powerful enough to distinguish these from benign false events.

Finally, we can also characterise and identify the sensors that are likely to be compromised, and calculate how many compromised measurements, or how many compromised sensors, a network can tolerate.

Corrupting artificial intelligence

But the risk doesn’t end there. To be useful to us, cyber-physical systems like driverless cars, or implanted medical devices, will use artificial intelligence techniques to learn how we behave and how the physical space around us changes.

The learning requires data from sensors, and as we’ve shown these can be compromised. Learning from this corrupted data could lead to our driverless cars, smart infrastructure and health monitors making the wrong decisions, with dramatic consequences.

This corruption is illustrated in the below diagrams of a machine learning algorithm, which classifies data into groups. You can see how the classification boundary changes when a single additional data point is inserted. In this case the point introduced seeks to maximise the overall error.

In this below case, which would be called a targeted attack, the introduced point seeks to make the red points be recognised as blues.

What stood out in our experiments was the low number of spoofed data points required to introduce fairly substantial error rates into the algorithm.

New attacks, new approaches

So far we’ve talked about the issues around compromised sensors. But there are many other issues that arise when we combine digital, physical and human dimensions in cyber-physical systems.

If this was just about the physical security, or just the cybersecurity, we’d be okay. We have the tools and techniques for reasoning about physical security, cybersecurity, and to talk about the trust we have in people. But we don’t really have techniques to analyse security for attacks that combine these three elements. So what do we need?

Firstly, with such attacks, we need to be able to perform risk evaluation in real-time. If some parts of the system have been compromised, what is the risk to the other parts of the system? Unfortunately, techniques for aggregating risk information don’t always scale very well, but research done within my group is addressing this through Bayesian techniques.

Secondly, we need to abandon the idea that we can entirely protect the system. Cyber-physical systems have much larger attack surfaces, and we should assume that the system will be compromised at some point. Instead, we need to develop the techniques that enable us to continue to operation in the presence of compromise to the system, or a part of it.

Thirdly, we need to design security techniques that allow us to combine the digital, physical and human elements. These all represent a threat for each other, but they can also complement each other in the protection of the system. The physical element can, to some extent, physically protect the digital and human elements. The human element can teach the cyber element how to behave, in order to monitor the physical space. And the cyber element can also monitor the behaviour of the humans involved in the system.

Success relies on trust, trust needs security

Artificial intelligence and the IoT have been much heralded as disruptive technologies with benefits that permeate society. Trust in these technologies will be the ultimate driver of their societal acceptance and overall success. If the systems are not secure, then they are not trustworthy.

Cyber-physical systems are already with us. We need to urgently address the security issues now to prevent loss of trust as we become more and more dependent on them.

Dr Emil Lupu is Professor of Computer Systems at Imperial College London. He leads the Academic Centre of Excellence in Cyber Security Research, is the Deputy Director of the PETRAS IoT Security Research Hub, and Associate Director of the Institute for Security Science and Technology.

Read Can we trust cyber-physical systems? in full

A post by Dr Deeph Chana, Deputy Director for the Institute for Security Science and Technology. This blog first appeared as an opinion piece on GATEway project website, 19 September 2017. 

The opportunities that driverless vehicles present are undoubtedly profound. None more so than the emergence of multi-modal transport services (trains, planes, automobiles … and boats) that will intelligently cooperate to take us from A to B without any human intervention.

Replacing the old biological controllers — namely us — the autonomous vehicle will excel in everything from energy efficiency to just being safe. The technology of today already affords us a near-term vision of the car where route planning and optimisation, refuelling and recharging, transactions with services (tolls , shops , parking lots), and authentication and hand-shaking for the purpose of site access control are all automatically achieved by the vehicle, without the human ‘in-the-loop’.

Removing the human from all of these piloting activities in concert, including that of physically maneuvering the vehicle, will prove to be the real transformation in experience that autonomy will bring to car users. The main outstanding technical piece needed to achieve this — the driving bit — is a problem that is rapidly being cracked by some of the largest and smartest companies in the world.

Removing the human from the driving seat

Furthermore, the use of artificial intelligence and deep-learning technology is poised not merely to deliver our replacement, but a significant upgrade. A ‘driver’ that will be better at learning, anticipation and adaption and one that will work tirelessly, around the clock. Driver 1.0 looks set, almost inevitably, for extinction. But, don’t worry if you’re feeling somehow obsolete, all of this will leave us with far more time to get on with the more important things in life like texting and motorway Tinder and will eliminate that potent source of stress, road rage — although there are no promises about the more general problem of rage on the road.

However, let’s leave the debate as to whether or not this transport paradigm-shift represents a psychological step forward for the road user for another day and settle for the fact that it certainly will be a technical leap-forward on how we go about the business of moving about.

Considering the comprehensive nature of the transformation we’re talking about, it is not unreasonable to ask if a re-think on what it means for a car to be secure and safe is motivated. Ironically, when we do pose the question, rather than the longer-term prospects of some kind of dystopian robo-world emerging, understanding how to be secure against humans emerges as the more pressing concern.

For whatever motivation — and there are plenty to choose from — humans are the most likely to seek the means and methods for compromising the whole operation; either by delivering costly nuisance cyber-hacks or by engineering complex orchestrated attacks that result in large scale economic hits or even the loss of life. Tragic incidents in urban settings around the world such as the most recent in Barcelona, illustrate how the car, even in its current form, may be used to generate terror and fear with global resonance and impact.

Paradoxically, the driverless car simultaneously represents an opportunity for virtually eliminating such incidents and the means by which their impacts could be greatly amplified. Both of these outcomes will be made possible by the unprecedented interconnectivity the car of the future will possess, where participation in a massive and distributed network of things including other cars, buildings, IoT devices, knowledge repositories and databases will provide access to huge computing power and a physical reach far beyond the individual car. Which outcome becomes reality rests on how well considered the design of this entire car-system will be to security problems and whether security will be ‘designed in’ from the start.

The argument that security is not the primary purpose of the car or that security incidents are generally not that likely to occur is a rationale that risks this aspect of the system’s design being given far less attention than it deserves. We might consider such arguments as rooted in the simplistic view of what we understand the car to be today rather than the reality of what it is about to become. It would be liberating and perhaps more in keeping with the technical revolution to consider the very concept of a car to be a fading reality, being replaced by a completely new mode of transport that bears only a superficial resemblance to the automobile. It may look like a car, move like a car, but in all other aspects it will not be one.

The Gateway project

Within the Gateway project — one of the UK’s autonomous vehicles urban demonstrators — we have been considering what security for driverless cars should look like in the near, medium and longer-terms. In the near-term we have examined the more practical aspects of securing vehicles that are being rapidly developed in the market by viewing our trial vehicles as moving cyber-physical systems: the driverless car is far more than just a moving piece of office IT. In the medium-term, problems such as ensuring that vehicles can trust connections to things around them with a digital pulse, including other cars, remains an open but tractable problem. Detecting security issues during the operation of such systems, countering problems in real-time and the legal ramifications of failure are all things that will keep our community and our wider networks working for some time to come.

Read The Security of Driverless Cars in full

A post by Professor Chris Hankin, Director ISST. This blog originally appeared on publictechnology.net published 19.06.2017.

With the cyber threat shifting its focus to sabotage rather than data theft, many of the defences deployed by public sector organisations will have to be adapted for the new world.

Information security policies are commonly guided by the CIA triad of confidentiality, integrity and availability. Many of the big security stories in the media relate to confidentiality, where data theft, for example, affects both individuals (eg. personal banking data) but also has a huge economic impact as a result of industrial espionage.

Integrity, or rather its loss, is most evident in the hijacking of websites by “hacktivists” seeking to deface content or replace it with political messages, but can also be associated with data, such as environmental monitoring, stock market trading or consumer price indices. Availability is often compromised by denial of service attacks as well as natural disasters, while recent high-profile ransomware incidents, when individuals and corporations are denied access to their data while a ransom is extorted, can also be counted in this category.

50 billion connected devices

The CIA triad addresses the security of information technology systems – our desktop computers, laptops, tablets and phones. Whilst these may represent the majority of the devices connected to the internet today, this won’t be the case in the future: there are currently around 15bn connected devices, set to rise by some estimates to more than 50bn by 2020.

This huge growth is fueled by the emergence of the Internet of Things in which connected devices control many aspects of our physical environment from home and leisure, autonomous vehicles through to city infrastructure. This move into the cyber-physical domain has already been presaged by the convergence of IT with the control systems built into our critical infrastructure and industrial processes. These systems were historically separated in both design and implementation, but economic necessity has driven them together. For example, the millions of lines of code which control a modern automobile has little separation between engine management and the car’s entertainment system.

New types of cyber attack

We have seen new types of cyber attack that are aimed at sabotage rather than data theft – such as the December 2015 attack on the Ukrainian power distribution network, commonly attributed to Russian involvement. The IoT hasn’t been immune either, with the emergence in late 2016 of the Mirai malware which targets machines running Linux and was used in the distributed denial of service attack (where multiple compromised systems, which are often infected, are used to target a single system) on the internet company Dyn. That attack was mounted through a network of Mirai-infected printers, domestic gateways, baby monitors and cameras (note, again, the lack of separation between consumer electronics and corporate and operational systems).

Many of the cyber defences deployed today will have to be adapted for this new world. It is unlikely that individual IoT devices (CCTV cameras, toasters) will have the computational power to run anti-malware software, and both safety and access considerations may militate against regular software updates. The increasing emphasis towards security-by-default (ensuring systems are set to the most secure settings) may help, but there is also likely to be a greater reliance on intrusion detection and prevention at the system level and a greater role for network monitoring.

These kinds of tools were traditionally based on recognising fixed patterns that are indicative of illegitimate behaviour, but there has been a recent trend towards tools based on anomaly detection. The latter tools use the artificial intelligence technique of machine learning to identify threats. Such techniques suffer from the so-called false positive problem – they may identify anomalies where none exist – but they are improving. Another problem is that it is sometimes difficult for the human monitor to understand why an ML algorithm has arrived at a particular decision. This is an important area for current research and we can expect to see rapid progress in this area. Since many artificial intelligence applications use machine learning, such advances are likely to have ramifications beyond the confines of cyber security.

The end of passwords

Another major area for change is in authentication. In the IT sector there is already the realisation that approaches based on strong passwords are not sustainable. GCHQ has produced more nuanced advice about passwords that recommends a number of simplifications. In future it is likely that other means of authentication will take on a more important and widespread role. Modern smartphones are already equipped with accelerometers (useful for gait recognition), fingerprint readers, microphones (voice recognition) and front-facing cameras (face recognition, retinal scan). A group of British universities recently developed a notion of cyber-metrics which supports authentication based on human-computer interaction, including measuring typing speed, pressure and interactions with a touch screen.

“In the IT sector there is already the realisation that approaches based on strong passwords are not sustainable”

It is clear the cyber world is changing and we can expect to see much more about cyber physical systems in the future. There are some exciting developments in the fields of artificial intelligence, particularly machine learning, and biometrics that will help to make us more secure. Expect to see rapid developments in the next few years as safety and security try to keep pace with increased user demands and technological capability.

Read How the Internet of Things poses fresh risks to public sector systems in full

A post by Dr Cathy Mulligan (Imperial College), Tony Kenyon (Guardtime) and Kacper Zylka (Imperial College).

Imperial College’s Blockchain research group (IC3RE) together with Guardtime have been investigating how distributed ledger technology – aka Blockchain – can be used to secure digitally-enabled critical infrastructure. Together they are providing an early warning system that embedded sensors have been compromised.

Cities around the globe are under increasing pressure to deliver high quality services to a growing number of citizens. Digital technology is being adopted – in what is sometimes called the ‘smart-city’ – to better manage assets, and deepen understanding of key services like waste, water, power and transport.

But as digital technologies become more deeply embedded into our urban environment they also create vulnerabilities to hacking, data manipulation and possibly breaches of citizens’ and corporations’ privacy. A recent report by HP outlined that as many as 70% of Internet of Things devices are inherently insecure.

This is a new threat landscape for cities to deal with. Traditionally cities have dealt with security from the physical perspective, but augmenting the civil infrastructure with digital technologies creates a lot more complexity.

Hacking of this digitally-enhanced infrastructure can place citizens lives at risk and cause significant economic damage to a city. At the same time, the contrasting lifespans of civil infrastructure and digital ones can further complicate issues. An average sensor currently has a lifespan of between 18 months and 4 years on average, but the lifespan of civil infrastructure can be significantly longer – from 80 years for housing to 150 years for a bridge or a tunnel.

How can we ensure the integrity of digital infrastructure in a manner which is future proof?

Security Issues in Smart City Sensors

We need to address multiple aspects of security when embedding sensors into infrastructure, including but not limited to:

1) Ensuring integrity of sensor state
2) Ensuring integrity of telemetry transmitted by the sensor
3) Preventing manipulation of telemetry/state while in transmission
4) Informing system administrators that a system may have be compromised

Working together with Guardtime, we investigated how blockchain could possibly assist with these security issues.

Blockchains

Blockchain is most commonly associated with the digital currency Bitcoin,  but the underlying principle is now gaining attention far beyond financial services.

Blockchains – a subset of distributed ledger technology (DLT) – allow untrusting parties with common interests to co-create a permanent, unchangeable and transparent record of exchange and processing, without relying on a central authority. See the UK’s Brackett review for more info.

It is beyond the scope of this article to describe DLT in detail but it is useful to note that in contrast to traditional means of storing transactions – databases – DLTs provide a historical transaction record – allowing users to see how assets have transferred ownership over the years.

System Solution Overview

Using Guardtime’s KSI solution , we developed a proof of concept called Blockchain-based Attestation for Industrial Systems (BAFIS)*. This is a hardware and software architecture that can be deployed in multiple facilities, and which performs continuous attestation of firmware of low-end embedded devices at the facility.

A user interface was also developed, which informed human operators when action needed to be taken after a possible system compromise.

The system does the following:

1. Registers IoT devices into the blockchain, (ID, firmware and configuration state)
2. Provides periodic verification of state, with alerts on tamper events
3. Provides an optional rollback to a known gold image or configuration

This is illustrated below in Figure 1:

Figure 1: System Overview of KSI-enabled smart city security solution

Future work

Our future work in this space includes some of the following open research questions:

1. Where possible, depending on volume and format, creating a process where telemetry off a sensor can be signed.
2. Verification of sensor data upon receipt at upstream data recipients.
3. Periodically verify data stored in back-end decision-making engines to assure data integrity over the lifespan of the data

* – Zylka, K., 2016, “Blockchain-based Attestation for Industrial Systems”, MSc. Thesis, Department of Computing

Dr Catherine Mulligan is a Research Fellow in the Innovation and Entrepreneurship group with a joint appointment to the Department of Computing where she is Co-Director of the Imperial College Centre for Cryptocurrency Research.  She is a Fellow and an Expert of the World Economic Forum for Blockchain Technologies.

Tony Kenyon is the CTO EMEA at Guardtime. Prior to joining Guardtime Tony spent over 20 years providing engineering direction for new products and solutions across enterprise, finance, and telco market.

Read The Role of Distributed Ledgers in Securing Urban Infrastructure in full

A post by Dr Silvia Ardila-Jiménez, Post-doctoral Research Associate, Imperial College London

The development of autonomous systems is one of the technology trends driving the fourth industrial revolution. Autonomous systems in transportation are perhaps the most widely talked about, but beyond this we’re already seeing systems deployed in sectors like environmental monitoring and agriculture.

The range of potential applications is huge: search and rescue, border surveillance, construction, energy, health, sports and recreation, agriculture, and food and water security to name a few. And whilst advances in this area are vast – fueled by machine learning, data science, robotics etc. – no man-made system can perform at the level of living organisms.

How do animals achieve such incredibly complex tasks and what are the biological principles that govern them? How can we use nature’s solutions for our own objectives? We’ve been assembling an international network of researchers to understand these fundamental questions; if you’re interested then please get in touch.

Flying animals can perform precise, agile manoeuvres, like hovering while feeding from a moving flower, mating in mid-air, and tracking and intercepting prey. Dr Tom Daniel (University of Washington and Director of the Air Force Center of Excellence on Nature-Inspired Flight Technologies and Ideas) is investigating how moths achieve these tasks which have very low probabilities of success.

Dr Daniel is building fundamental understanding on how they integrate multiple sensory modalities (e.g. vision, inertia), which may help to improve flight control in engineered aircrafts. The mechanics of the moth model may also provide insight into alternative modes of sensing and actuating.

Robber flies are relatively small insects, but have evolved flight strategies which make them as successful predators as larger flying insects. Dr Gonzalez-Bellido (University of Cambridge) is investigating specific anatomical adaptations in their visual system that enable them to track small targets, and adjust their flight trajectory to intercept their prey with high-accuracy.

Birds have evolved their own adaptations for flight, dynamically adjusting their wing shape in response to wind. The Windsor group at Bristol University record and model the dynamic 3-D structure of bird wings during flight. They’ve shown that this knowledge can be applied to make drone wings which respond to the environment and improve flight.

At Imperial College, Dr Mirko Kovac’s Arial Robotics Lab has combined the capabilities of two different organism types to develop an aquatic micro-air vehicle for monitoring water health. The robot, as seen below, dives into water like a gannet, and then launches like a flying fish back into the air. You can see a video of this in action here.

Image copyright Ben Porter

Whilst flying animals are an obvious area for inspiration, worms can also give us new solutions. C. elegans for example are tiny nematode worms that burrow through soil – a complex chemical environment – in search of food. A robot inspired by this worm and capable of navigating obstacles with minimal sensing has been developed by Dr. Boyle’s group at the University of Leeds.

Social animals bring a whole set of useful and challenging behaviours. Dr. Paoletti’s group at Liverpool is looking at swarming and schooling to develop groups of robots that can collaborate locally to perform tasks such as recognition and surveillance.

Engineers taking inspiration from nature, often called biomimicry, is nothing new; Leonardo Di Vinci’s flying machine is a famous early example. With today’s technology however we can go beyond merely mimicking nature; we are capable of looking deeper at the underlying natural principles, and adapting them to improve our own systems.

The challenge now is to bring together expertise from engineering and biology to study, understand and assess the potential benefits of looking at nature for inspiration to enable improvements in application technologies. This is something we’re actively engaged in at the Institute of Security Science and Technology. If this is something you are interested in please get in touch!

Silvia is an engineer with a PhD in computational neuroscience from Imperial College London. In her Ph.D. she looked at how different areas in the primary visual system interact to process incoming information using large data sets. Silvia is currently working as a Post-doctoral Research Associate in the Department of Bioengineering and the ISST working on pathways from nature inspired research into application technologies.

Read Worms, birds and insects inspire the robots of the future in full