Hacking Smart Meters, Single Chips and Updating
- Published
- in Smart Energy
This week was an interesting one for smart metering announcements. Accent – a Franco-Italian semiconductor design house announced their smart meter on a chip, prompting Jesse Berst of Smart Grid News to enthuse that the “Smart Metering Business has just changed for ever“. Sorry Jesse, but I don’t think so. Elsewhere, in Providence, Rhode Island, New England hackers were convening at QuahogCon to discuss the security of standards. The two announcements provided a good demonstration of the gulf between the promoters of smart metering and the reality of the state of the standards they intend to use. In the same week, ZigBee closed its call for comments on the Technical requirements Document for its Smart Energy Profile, giving the impression that the standard is not far from completion.
The gulf between the enthusiasts and realists is wide. It is worrying that much of the industry is rushing blindly towards deployment, with little understanding of the risks and what can be done to mitigate them.
One of key mantras I keep on hearing repeated when security of the smart meter is raised is “why would anyone bother to hack it?” Josh Wright, talking about ZigBee security at QuahogCon hit the nail on the head when he answered that. “As an attacker, ZigBee lets me interact with the real world – that’s exciting. I can interact with a dam, or natural gas distribution lines. We’re looking at a wireless protocol that lets us interact with real things in the real world – it’s not just credit cards.” The industry forgets the excitement that comes from “because I can” and “real things” And it only needs a few people doing that to fuel scare stories that will kill the whole industry.
Let’s start with Josh. He’s been looking at ZigBee security for some time and reporting on the issues, both of the standard and also of implementations. Like many other wireless standards, ZigBee started off with fairly basic security. Over the years they have added more and better ways of implementing security. However, although these exist within the specification they’re not generally mandated and most ZigBee products on the market don’t implement all of them. Even where they do, there may be errors in the protocol stacks which allow hacking attacks to succeed.
ZigBee’s not alone in this – every wireless standard has gone through the same process. However, the issues only come to light once a standard starts to gain market traction and ship products in volume. Hackers generally don’t bother with obscure or academic standards – they concentrate on the big shippers. Up until now, ZigBee has not been widely used and has escaped scrutiny. With its move into the big time, it’s starting to attract that attention. And that attention is showing that a lot of what is in the market is seriously deficient.
You can see Josh’s presentation and also listen to it. If you are working in the smart energy industry, I’d urge you to find 45 minutes to listen. It is both entertaining and informative. And scary. Robert Cragie has posted some pertinent points about this presentation on his Gridmerge blog.
Last month the ZigBee Alliance asked for comments on its Smart Energy Profile 2.0. It’s part of a new “open” process. I hope that openness will involve publishing all of the comments, so that the industry as a whole can help scrutinise and drive the specification forward. If you missed the deadline, you still have the opportunity to post comments on the draft specification itself, although that round closes on June 4th. If you do submit comments on any security issues, I would urge you to make them public as well, to get wider scrutiny.
ZigBee is by no means alone in this debate. None of the low power wireless contenders vying for this space have had any real degree of external analysis. That’s equally valid for Z-Wave, Wireless M-Bus and Bluetooth low energy. All have written credible specifications, but none have been independently tested. ZigBee is doing the right thing by being open about it, and the others need to follow that example.
Which brings me to the single chip. If that seems an odd jump, it’s not, because the single chip suppliers are selling a dangerous belief, which is that you can make your meter cheaply now, taking advantage of the high levels of integration that are possible, and then upgrade it later. The “jam tomorrow” story of upgradability is a very dangerous sticking plaster that is starting to be used to cover up any worries about security.
Single chips will almost certainly be the way forward. It’s the natural evolutionary path for electronics. But it’s one that is normally taken after a standard settles down, particularly if portions of the standard go into ROM. Because it’s driven by cost reduction, it normally means that the resources on the chip are limited to what is needed today, because unnecessary resource costs money. As a guarantee against future problems, it’s a promise that doesn’t work.
The best specification for a smart meter that I’ve seen is the British Gas one. It mandates upgradeability. It also mandates a recovery procedure from a failed update, which is excellent and more far-sighted than most. And it specifies a product lifetime of twenty years. That’s nothing new for the metering industry. But it represents multiple lifetimes for the wireless industry. And that discrepancy is where we have the problem.
It’s informative to have a brief history lesson in order to understand what twenty years means. If we look back to May 1990, the more advanced of us were running Windows 2.0, although many were still using DOS, or even GEM. I’d recently bought a 286 based PC with a 20MB drive, thinking that would be all I ever needed. No piece of software I buy today would run on that PC.
Digital Mobile telephony was still a dream. No mobile phone from 1990 would work today. Five years later I was lucky enough to own a recently launched Nokia 2010, which still works on the GSM networks. But unless you lived in Europe and were at the bleeding edge of GSM technology, then any phone you might have kept from 1995 is unlikely to work anywhere in the world today.
In terms of short range wireless, everything was still a dream. In 1995, Wi-Fi was seven years away. It’s pre-cursor – 802.11 would arrive the following year, but operating a different frequency, so any PCMICA cards you might have bought then would not work with anything you could buy today. And ZigBee, Bluetooth and 802.15.4 were nowhere on the horizon.
Go back just ten years and there’s not a lot of difference. Most PCs were running Windows 95 or 98 and the first Bluetooth devices were on the market. If you had one of those, it would still probably work with a current Bluetooth product, but at a base level of interoperability. The original security features would still be working, but the more recent enhancements would not. The first Wi-Fi / 802.11b products were beginning to emerge, but with security limited to WEP. They would probably connect with current Wi-Fi products, but it would not be possible to upgrade them to work with the higher security of WPA, which is deemed essential today. And ZigBee was still nowhere to be seen.
We need to regress a mere five years to 2005 to find the first ZigBee release and products. They would be largely incompatible with today’s ZigBee PRO products and could not be upgraded to the security requirements of either of the Smart Energy Profiles.
Bluetooth and GSM have fared best in terms of long term compatibility and security. Whether that is down to luck or judgement can be debated, but that’s not the point. We forget just how quickly technology moves, and how, as it becomes endemic, hackers find flaws either in the standard or the implementations.
The bottom line is that none of these early products can be upgraded to include what is considered best practice even five years later, let alone ten, fifteen or twenty. And there’s the rub. Upgradeability might allow a small improvement to security, but any major flaw is likely to require more resources than a highly optimised single chip contains.
This is a fundamental factor of the rapid progress of technology and the smart meter business needs to understand it. For the future of the industry we need to ensure that the security is as well tested as possible before we start deployments. There are a lot of companies that want to ship in order to make money today, but that puts the entire industry at risk.
The other concern is that wireless upgrades are difficult. I’ve been working in wireless for many years and my advice to customers is never to upgrade a product in the field. With each upgrade a percentage of products will probably fail and that percentage will grow with each new upgrade. Upgrading is not a panacea for cutting corners in the original design; it’s an open cheque-book for future support costs when the upgrade inevitably falls over. It’s a subject that is glossed over, but sufficiently important that I’ve devoted half a chapter to it in my forthcoming book – The essentials of short range wireless.
We must not forget that throughout the history of metering, people have convinced themselves that there are ways of fiddling meters to falsify the readings, from rewiring them, adding magnets, inserting photographic film to act as a brake, using vacuum cleaners (gas meters) or magic devices that alter the spikes in your supply. Once we add digital electronics and a wireless link, this band of amateur hackers will grow from a trickle to a flood as every engineering student tries to find a way of reducing their bill, or turning off their lecture hall lights.
There is no perfect solution – it will become a cat and mouse game, but we need to start from a position of strength. Rushing to market is more likely to be a deployment of weakness, from which it will be difficult to recover.
There is no inherent reason why a single chip should be any more of a security risk than a multichip implementation. At the technology matures it may even be more secure, as if firmware is hard-coded in ROM it removes the opportunity for the more aggressive hacker to suck it out of an external flash or EEPROM memory.
I should make clear that when I’m talking of single chips for smart meters, I’m referring to a single chip that performs both the communications and the metering function, so that will typically be a chip with a radio, and a number of processors running the baseband, protocol stack and metering application. Different companies architect these in different ways, so it may include multiple independent cores, or a single more powerful one. But the rationale behind a single chip is invariably to get to the lowest possible cost, which limits the flexibility that is included in the chip design.
Once a market is mature, there’s a lot to be said for single chips. When it is not, then rushing to a single chip to get the lowest possible price risks making architectural decisions that set things in stone too early. Here the problem becomes one of mindset. If a company has put significant effort into producing a single chip, there’s a large barrier to throwing it away and starting again when a non-patchable security flaw arises.
At this stage of the industry’s development it’s much safer to consider using separate chips for the local radio link and the metering application. That makes it a lot easier to change or upgrade either when the need arises. As the standards covering these two areas may not be developing in parallel, either can be changed when required. In contrast, single chips tend to wait for all parts to be updated before revving the silicon. This still poses a problem for units already out in the field which do not have the base capability to run an updated standard, but it makes it much easier for companies to react and update their products to the latest best practice.
Am I correct is assuming that single chip is more prone to security hack than a two chip smart meter.
In single chip if the only CPU is hacked, the hacker takes over all communication and control. But for 2 chip, a hacker can take over the communication chip, but the control is still secured?
Travis’ blog is at http://travisgoodspeed.blogspot.com/2010_01_01_archive.html. Details on his probe are at http://frob.us/projects/syringe/
And long live the 8051. I’ve been wondering for the last ten years or so at what point there will be a wireless equivalent to the 8051, that can bring short range connectivity to products in the same ubiquitous manner? Maybe that’s something for another post…
You’re right to highlight the issues as pointed out by Josh Wright and others. I responded to some of the issues they raised in this article: http://gridmerge.blogspot.com/2010/04/i-was-reading-article-in-mit-technology.html.
I like your anecdotal look at technology of yesteryear. However, if you look at embedded systems, 8051-based systems are still being specified today and this device was in widespread use 20 years ago. So it may be the case that the embedded workhorses of today may still be in use in 20 years time.
Nick. As a software security luddite I am following the smart metering security discussion (wired & wireless forms) and the various protocols being touted with great interest – trying to decide who to believe and filter through to what I/we can understand and apply. It’s not easy but articles such as yours do help me keep a clear head on following the discussion intelligently.
It’s not just smart meters and those physical things you mentioned at risk. There are home automation systems, health care devices, and various security, media and communications ‘services’ drifting over to distributed networking and comm’s with ZigBee, Z-Wave, Bluetooth, even DNP-3 and others in what seems to be a very blasé manner.
A company I’m involved with made a conscious decision to use 2.4GHz WiFi* because it’s IP, has mature’ish security and can easily migrate between wired IP when available. And, to specifically avoid ZigBee and others where – as you point out – the rate of change and development is still too fresh to put into ‘secure metering and HAN applications’. (How NIST succeeds in specifying reliable implementable models either side of the industry meter remains to be seen).
There’s one thing I want to ask you to reconsider, that is the hype about smart meter-comm’s-security-program upgrade-ability, and what it really means in this context of smart meter and grid infrastructure asset life. In the case of UK you have BG coming out with a fairly well documented specification, albeit overly proscriptive in some places – meter & HAN/IHD hardware and operation – and loose in others – systems & policy.
They made statements about upgrade capabilities and 10-20 year lifetimes which you question, I think, as being beyond reasonable expectations, given that wireless and networking have evolved so rapidly in only the last 10 years. That change rate is true, but it’s not worth worrying about in this context because the nature of the upgrade pathway they want to preserve for ~20years lays in the ability to modify existing low level functions, variables and communicate with a meter, transfer data to or through the meter, perform a few algorithms. So, as long as important firmware code isnt locked up in ROM (as you say) and can be patched OTA as needed, then that satisfies the life time goal. There’s no foreseeable issue with closing RF spectrum – say 2.4GHz – or doing away with underlying coding and protocols that prevents retaining a system such as this for 20 years or more.
There’s still high volume X.25 used in niche app’s for example, and that’s been around decades, and the POTS is still operable as a full service device in a very crowded space over the same wire pair. It’s not very secure at all but life goes on in spite of that.
The key is not to over hype the expectations of smart grid devices such as meters and load controls and for the sake of the security issues, to focus on maintaining an upgrade path for security & integrity.
Nice post, Nick. Security is a big issue in any type of wireless and guys like Josh Wright and Travis Goodspeed are pointing out that it shouldn’t be taken lightly. Upgradeability is good but the problem occurs when a critical flaw in hardware is discovered. Check out Travis Goodspeed’s blog for more info on multiple hardware flaws. An interesting one is the side channel attack of a radio IC via hypodermic needle to extract the AES keys.