I’m talking about this at #iotlondon next week.  Objective: to let people know what I’ve found so far and see if others want to collaborate.

It all started when my router blew up because I had 16 devices talking like the clappers to multiple services.  Got a new router but that’s no permanent solution.

I thought: we’ve got all these devices with their own protocols talking to all these cloud services also with their own protocols.  We need a hub in the middle to make sense of this.  But, this hub needs to be powerful, versatile, low-cost and low in power consumption (as it’ll be always on).

That combination of attributes turns out to be challenging.  I’m looking at a number of hardware and software platforms and have some interim results.  Also building some demo apps to apply a little stress testing.

This hasn’t been plain sailing.  Glad I gave myself a deadline for this talk.

The quest continues.  Anyone got any spare coconut shells?

8am start in London means I’m up no later than 5:30.  Or was I dreaming?  No, the Quantified Self “movement” is gathering momentum.  In London this is thanks to the QS Meetup led by Adriana Lukas, one of the speakers at this session.  Great that NESTA picked this as hot topic.  They even posted a video!

Because of the document referred to in the second link, there is little I need to say about the talks themselves other than that they were very well done. In the ensuing questions after the talks some issues cropped up that are dear to my heart (with my “Internet of Things” hat on).

Architecture:  Many people seems to be developing architectures but there’s little sign of an emerging de-facto standard.  This is for the usual reasons.  Product vendors (Adriana cited Fitbit) produce end-to-end “stovepipe” solutions that sit in “walled gardens” that they control.  This allows them to get continuing income from selling us back our data to help fund product development (goodness) at the expense of making integration between systems extremely painful.  We need an architecture that’s vendor independent and based on open data principles, with privacy controls.

Tools:  The situation may be better than people think as there are lot of good tools around.  These tools are not, however, well suited to people who just want to use the data.  For example, getting  data out of a proprietary system and uploading to somewhere else for  analysis requires a fair amount of programming skill and not everyone wants to learn that.  This may be inevitable at this stage of market development.

Gateways:  Given that we’ll be facing this stovepipe problem for years to come and tools may be too technical for most users, it will be useful to have devices that are designed to translate formats and protocols straight out of the box.  With colleagues, we are looking into what could be done in this area.

Privacy/ Security:  As was pointed out in the talks, this is fairly fundamental if we are talking about personal data.  We need to be able to access our own data (all of it), grant access to others with a legitimate need and feel confident that unauthorised access will not occur.  Part of this is technical; there are well-proven mechanisms for encryption, access control etc.  The commercial, legal and cultural issues are more complex.

“Bill of Rights”: Attempts to codify appropriate behaviour in relation to data are underway. At the Open Internet of Things Assembly in June this will be on the agenda and there’s already an active googlegroup on the subject (iot-open-data).  It’s too early to say how this will progress and to what extent it will be adopted or enforced.

Data Analysis: This is a tougher challenge as the needed techniques will vary widely.  A good starting point is to look at analysis techniques in other fields of endeavour to see what can be re-used;  The case studies presented at regional QS groups should yield ideas on techniques and tools to implement them.

I look forward to further QS sessions and will continue to engage as much as I can.

OSHUG, the UK’s Open Source Hardware User Group, goes from strength to strength so it was a pleasure to attend yesterday.

This was a special edition that included the first Raspberry Pi meetup.  The conversation went round the houses but was none the less useful.  Whilst this device is coming in an an attractive price it is currently lacking in an ecosystem to rival Arduino and MBED.  It is also suffering from an acute shortage in supply; RS guys were on hand to explain how this is being addressed.

The ecosystem discussion was interesting.  It may evolve organically.  However, IMHO some strong leadership is needed to ensure that this happens in a coherent fashion so that a collection or coherent and interchangeable building blocks evolves.

Once #OSHUG proper got started we have 3 great talks:

• Open Compute: interesting to those who may need to build server farms.  In direct contrast  to Google’s approach of keeping the tech to themselves.   This is more large scale engineering than hacking, even if it started that way. DC power distribution is interesting – that could find wider application. Chris Swan probably works for the smallest kind of company that would need this.

• Beagleboards, and especially BeagleBone has matured into a really interesting platform now.  Crucially, there is a comprehensive ecosystem around it.   This looks ideal for home hubs and such like projects.  I’m particularly impressed with the IDE options and software that comes with it.  Sorry, Raspberry Pi, but even with the steep cost differential this looks like better value for money, especially if you value your time. Amongst the demos Roger Monk brought along was a BeagleBoard running Android!  Looking forward to getting my hands on one.

• Henk Muller gave us another talk about XMOS.  This differed from OSHUG #1 in that it was more about concurrent software execution that XMOS’ rather unique hardware.  Interesting if you have the sort of project where highly predictable timing is important.

Thanks again to @9600 for impeccable stewardship of the group and to #C4CC for hosting and the regulars for showing up.

First and very important after last time, the organisers had found an excellent venue in a convenient location and they even provided free beer. What’s not to like? This is really essential when you have great talks – which again we did.

First up was @andrewlindsay talking about the gateway software he’s created for Sukkin’s wireless hub device. This can handle arbitrary numbers of wireless sensors communicating at 433 or 868 Mhz. Thanks to an included socket it’s also amenable to Xbee or XRF (ciseco.co.uk) wireless connections. It’s MBED based so there’s enough space to do the software properly, something that’s almost impossible on current Arduinos.

At 2 watts, the device is eminently suitable to being always on, something that many boxes are not. Being open source we’ll never be limited in the directions we could take the software that Andrew is creating.

Andrew has only tried a few sensors so far but, interestingly, these include air pressure and dust particles. He’s using Jeenodes at the remote end.  He plans all sorts of further enhancements, notablly a web-based configuration facility.  Excellent stuff!

@rollinson (Jeremy) then talked about “Open Telematics”, a mobile (currently Android) platform for telemetry that posts data to pachube. The first outing of this is a vehicle app that hooks up to the CAN and OBDII buses now legally required in all cars.

The platform uses the phone for GPRS, accelerometer, GPS and screen and hooks up to the CANBUS via a proprietary dongle. I want one. I’ve signed up for their beta programme.

At Jeremy pointed out, this is unique in that people will be able to get their own data, something not possible with commercial vehicle tracking systems. Datastreams will include, at least, fuel flow, rpm, speed etc. As well as offering the app for a reasonable price they have plans to make it social and add premium functionality. However, as we can access the data there’s nothing to stop others from innovating around this. Apparently Open Energy Monitor and already looking at how they might integrate this with their system.

So, I thought, good job I did not go to the trouble of building this. It does the job and the price is right.

Third session was from Cesar Garcia Saez and colleague from Medialab Prado Madrid. This sounds like a fabulous facility where people can come in and work together on interesting projects.

It’s open to all and inclusive in bring togther a wide range of disciplines from art to technology.  It’s also inclusive to people who are not Madrid-based through the use of Wikis, video streaming and regular open calls to bring people in from other places. Something like 50-100 collaborators are involved.

As this is government funded, one of the rules of play is that the results are open sourced. The projects they do at the Medialab tend to be pre-competitive, not products ready for market so this open policy is not a problem.

Their “Smart City” project was given as an example. In this they collected air quality data and displayed it on the side of a building. Following up on this they are getting involved in the Air Quality Egg project (now active on kickstarter).

All in all an excellent meetup. Many thanks to Ed and Alex. Keep up the good work.

Ecobuild came around again this week and I braved crowded tubes to make it down to ExCel. If you needed a solar panel look no further. I did not. Instead, I was there to see what was happening on the energy gadget front. In particular, I was looking for energy monitors and heating controls that I could recommend to friends and associates in our energy coop “Low Carbon Chilterns”.

In a much earlier blog I mentioned my fortunate chance to evaluate one of the first advanced heating controllers dubbed “Intuition”. This has now come to market under the OWL brand with numerous improvements and has become something that no home should be without.

We used to have a “state of the art” system with wireless thermostat and multi-period programmer – or so we thought. However, through careful monitoring of gas used I discovered that our system was far from optimised. With continually rising costs this was getting expensive. The main reasons is that most of the currently-available programmers are not designed for humans to use. Three buttons pressed in some hideous combination allow any setting of time period. Our programmer not being linked to the thermostats we only had one temperature setting for hot water and one for air. Moreover, this programmer is so tedious that most people do not set it optimally or change it as often as necessary (some don’t change it at all). This is especially the inefficient in Spring and Autumn when outside temperatures are volatile.

Management of an Intuition system is a breeze. You have a web-based User Interface (my preference) and controls on your smartphones. The fact that you can easily change any setting makes a huge difference. This is pretty fundamental to modern product design. Machines should make our life easier not try to enslave us (think last-generation video recorders).

You don’t want to keep the water shower-hot 24.7 just in case you might feel like a shower. Instead, have it hot when you know you will need it and any other time just punch one button to bring it up to temperature.

You may have heard of the “optimum start” feature of some modern boilers. Intuition’s version of this is based on your settings that say something like “give us 45 degrees for ours showers at 6am M-F and 8am on weekends”. The system looks at the outside temperature and calculates when to turn the boiler on and off.

The latest version of this system is also extremely easy to install, it simply replaces your existing thermostats so anyone with basic electrical skills can do it.

The net effect in our case has been a significant saving of energy. Since heating accounts for 80% of energy used domestically this is something that everyone should pay attention to. In our case the new heating controls are only one of many things we have done to effect a 30% saving in our energy bill. However, I would attribute at least half of this to accurate control of heating. This constitutes a sub-one-year payback with the current model.

BTW. A modern boiler is a pre-requisite that I sometimes take for granted. Old ones may be more reliable but they are rarely efficient. Make sure there’s a “diverter value” – not all systems have the ability to hear just the air or just the hot water. You need a plumber to install one of these, hopefully you have one already. With this in place you have separate control.

Back to Ecobuild and some other products that caught my eye as I rushed past all the panels.

Energy monitors are coming of age. I don’t subscribe to the idea that a little box on your windowsill will give you enough information to justify changes in energy-use behaviour. Put simply, you’ve got to have graphs of energy vs. time. CurrentCost were first to market with affordable web-based electricity monitoring and I’ve been using theirs for some time. What I always wanted first and foremost was gas so for that I had to build my own. However, electricity monitors are widely available and inexpensive considering the savings they can produce if properly used.

Now, a number of other suppliers are jumping into this, OWL and Energeco being two displaying their wares at the show. The OWL device, also dubbed “Intuition” is a neat little box that adds onto the familiar OWL monitor. This looks particularly interesting for installations with PV as well as grid power. I will evaluate this ASAP.

Another interesting type of gadget is one that diverts any excess energy from a PV array into heating hot water.  This is a relatively new category and I confess not to understand exactly how these work.  Something to watch out for.

As ever, watch this space for thoughts and developments on energy gadgets.

In the beginning (April 2011) there was a beta apps platform (announced at the hackathon) that allowed the creation of apps that other users could use with their datastreams.  This has now been superseded and a new facility is being tested prior to launch.  This is based on OAuth as the way of allowing a given app to use one’s data.

The new process for the user

• Go to the app’s home page
• Click on the install link
• Authenticate at pachube
• Configure the app, associating the user’s data feeds
• App is now ready to use

How it works

The diagram shows a four step process for the user.  The grey boxes are the app and the white ones are pachube.  R shows redirects and dotted lines show server-server communication.

The pachube OAuth API spec shows how to implement this.  I did a version in PHP which took an hour or so to implement.  This is currently specific to my apps but I plan to generalise it in due course.  It’s not the whole solution because the configuration piece will tend to be app-specific.

Associating data to your app

This used to be handled by the pachube apps installer which is no more.  This is no great loss as the original UI, being generic, could cause user confusion.  Now that the developer can roll their own UI you can make it as useable as you like.

A first demo of this approach

Here’s an example.  After authenticating with pachube I take the user here where they can select the feeds and datastreams to use.  This needs explanations of course but the important feature is that both the app and the data semantics are visible together and clearly related.

Once the user confirms this configuration we can save it and take them straight to the app itself and set it running.  It this point there is no data store within pachube for the configuration.  Again, not a problem as you are free to store this wherever you like.

Limitations/ outlook/ suggestions

The current OAuth mechanism gives an app access to all the feeds belonging to the currently logged in user.  This is fine for now.  However, I foresee that users will have certain feeds and/or datastreams that need to be more private than that.  I understand that this is in the pipeline from pachube. It should be quite easy to retrofit this capability when the time comes if you cater for it in your design (each datastream will need to know what access the key provides);

Another issue is that each app needs to replicate the fields in pachube’s “App” entity (description, urls etc).  It would be useful if this could be accessible via an API so that the two could be sync’d (using the app owner’s master key).  Ideally, the app should be able to instantiate itself within pachube using an API rather than manually – this would avoid any accidental transcription errors.

The current (beta) versions of a couple of apps are at https://pachube.com/users/paul_tanner/apps.  I have asked how you get apps onto https://pachube.com/apps (ie more generally available).  No answer on that while the overall facility is still being completed.

Watch this space for more info as we test this more fully.

Quite a painful upgrade from WP 2.0.  I should not have waited until white screen problems came along.  Have imported historical content from a backup saved in RSS form. There’s more to do with the theme, blogroll etc.

As technology has advanced, we seem to have lost something important along the way. We used to be creators of technological things but there’s now an increasing tendency for us to be consumers. Back in the 70s if you wanted a computer you had to make one. At least by then there were LSI chips and kits like the one that famously inspired Bill Gates and Paul Allen. Making things was an excellent and inspirational way of learning about technology. Increasingly now, we buy the tech we need. This is a lot less fun and it leaves a significant gap between the way we’d like things done and the devices we have to live with.

For some reason the desire to make what you can’t buy appears to be gradually returning, as evidenced by the advent of “hackspaces”, “fablabs” and home hacking in general. This is despite the resistance created by health and safety issues associated with tools, soldering etc. Groups like “Open Source Hardware User Group (OSHUG)” are appearing.

I was glad to discover a BBC program on this on Radio 4 recently (hopefully still on iPlayer by the time you read this). This contains coverage of the excellent London Hackspace (well done guys) and a reference to the “FabLab” work of the MIT Media Lab’s Bits and Atoms programme, capably explained by Neil Gershenfeld in this TED Talk of a few years ago.

One of the first things that the Beeb commentator had to explain was that the term “hacking” originally referred not to acts of greed or malice but simply making things work in ways that were not originally intended – “just for fun” (also the title of Linus Torvald’s book”. See MIT’s gallery of hacks for some examples of old-style hacks. I guess both of these meanings will inevitably stick but, to us, hacking is benign.

Mobile App development is one of the few areas inspiring the digital generations to innovate. I attended this year’s “OverTheAir” “hackathon” at Bletchley Park recently. Great venue, interesting talks and hundreds of people hacking interesting stuff over a 2-day event. (Visit this venue anytime to see some fabulous tech history.)

Just as I was thinking that I had all the mobile app I’d ever need, Google made Android into a platform for physical device innovation with their Android Accessory Development Kit (ADK). It’s early days for this architecture that bridges the mobile world with that of electronics and I hope to see some interesting innovation around this. Our hack at OverTheAir make use of this to build an “Internet of Things” device based on an Android phone. That became the orb mentioned on this blog recently.

I really do hope that this dying art of the the “maker” is in a sustainable revival. I particularly hope that kids will get the kind of support and encouragement that we did back in the 60s when making things was a more respected pastime. “Maker Faires” that attract young and old makers alike are cropping up across the globe. It would be great to see a return within the educational system to practical science teaching that inspires, enables and supports innovation.

This is going to be a long-running project as there are so many ways to do this. The objective is to provide an unobtrusive yet effective way to communicate with people around a home or office. The idea is not new and you can buy one of these from Ambient Devices. However, this one is quite different in the way it gets its information. Specifically, I wanted it to be pachube-compatible so it can be used in numerous types of projects where people have full control over the back-end processing and several choices of communication medium.

The specification(s):

• low-cost components
• on-board processing possibilities
• wired or wireless options
• standards-based connectivity
• option to run on battery power
• brightness control

I made one to evaluate the idea:

• glass dome (IKEA or B&Q)
• base unit (painted wood for prototype)
• Microcontroller (MCU) and comms board stored in base
• Power LEDs (RGB), drivers and heatsink (yes, even LEDs get hot)

Output devices/ actuators should not poll as this just chews up power; feels wasteful and is. I chose the TCP Socket protocol provided by pachube. This features a publish and subscribe mechanism so the MCU does not have to keep polling for data. Instead, it is alerted when the data changes.

The candidate MCUs were AVR (Arduino) and MBED, the latter being chosen for the first version because the socket libraries were available and there’s plenty of RAM allowing for communication in JSON format. This one uses wired Ethernet. Another version will most likely be built with a Nanode. This will allow a cost saving at the expense of more work on the software. One interesting option would be to use a low-cost wireless receiver instead of wired Ethernet. I also made a version based on a Seeeduino ADK board that communicates via an Android phone. I’m going to try that with an old Android I know longer use. Also with an old USB dongle if I can figure out how to drive that. I was pleasantly surprised to be able to use the Android phone as a gateway although this needs two background async processes – a bit complicated.

BTW. Many thx to @dcuartielles or getting me off base with ADK and to the guys at #londroid for pushing eclipse as the way to go for Android dev.

The LEDs were a tricolor power LED module (LX1610RGBW/A) and the driver was in a quad DIL package (DS3658). For simplicity, the first version limited the current to the LEDs so was not very bright. Subsequent versions will have a larger heatsink and, optionally, be able to run these LEDs at full power. A current-control circuit will probably be needed to do this safely, given the risk of doing everything through software and accidentally over-driving the LEDs.

The firmware is made simpler by using the MCUs pulse-width-modulated (PWM) outputs to control the relative power to each LED. Most of the work is in setting up the connection to the server and dealing with data as it arrives. As firmware is easily loaded from PC or Mac we made a version that cycles the colour values while testing.

In order to control this while testing we made a web page with a colour table selector that sends colour settings to pachube.

In real applications one would take sensor data and process it before sending an appropriate colour to the orb.

Watch this space for future versions and applications.

I had been thinking about this and was spurred into action by this item from Horizon Digital Economy Research. I have commented on a lot of articles about smart meters and decided it was time to set down some thoughts. As I was putting pen to paper another article arrived from the Guardian with a number of interesting points.

Smart meters are coming our way, despite any objections such as those being raised in the US (they are really worried about radio – or is it political?). In the medium term the data derived from this will not only reduce meter reading costs and improve billing but will become the lifeblood of a new smart grid.

For reasons best known to the utilities, the chosen data transmission system is to use the GSM (mobile) netwrork and send data at low (30 minute) resolution. Consumer access to this data is not guaranteed and there are suggestions that, at best, it will be available in some digested form on utility company websites. There is an obligation for them to provide in-home energy monitors but these are unlikely to be any more effective that those currently in use. (Initial interest quickly wanes, as do the savings.)

This may make the utilities comfortable but it does not provide a basis for ordinary, non-fanatical, consumers to save significant amounts of energy. Given the enormous cost of the meter rollout, this is at best a massive missed opportunity.

I say this after two years’ study of the home energy-saving challenge in which I and a colleague have achieved savings of 25-30%. This is the kind of saving that is and will continue to be only available to enthusiasts who are prepared to sink a lot of time into the challenge.

To enable the majority to achieve similar savings we believe that the following is necessary:

1. Data must be available instantaneously and must have a resolution in the order of 20 seconds or better.

2. Data capture must be fully automatic and accurate (unless the consumer is super-fanatical enough to do it manually)

3. Data must be available to consumers for their own use or to release to third parties at their discretion.

4. Social tools are needed to help people to support their peers on their journeys to home energy efficiency.

The social tools will come (from 3rd party sources) as and when the data are available to drive them.

The above are blindingly obvious to anyone who has made a serious effort to understand their energy consumption. We hear a lot of complaints about tariff complexity but that’s nothing compared to the problem of figuring out whether your boiler or freezer are delivering reasonable efficiency. The most important devices to understand are those for heating and cooling that normally control themselves.

This is where frequently-updated data comes into its own. Measuring the daily consumption of a boiler or a freezer tells you that it’s consuming too much but not why. However, with more detailed information such as precise on and off times you can decide what to do about it.

Not only does the current plan (a spec would be nice) seem to meet none of 1 to 3 above, it will also further exacerbate the already severe congestion of the GSM network and do nothing to advance the need for ubiquitous, fast (wired/ cabled) broadband in this country. It’s hard to get to the bottom of the decision to use GSM. Is it data security paranoia on the part of the suppliers or is it fear that incompetent installers would mess up anything not wireless?

This looks like yet another episode in the sad saga of government-led projects that either over-run financially, fail to meet their objectives, get cancelled or all three.

.. and they call this Smart?