Post date: May 02, 2015 6:43:21 AM

A story of home automation with openHAB, Z-Wave, and MQTT

A friend of mine had a bus system installed in his apartment twenty odd years ago to control the lights, and I was suitably impressed. He raved about being able to switch on the light in the loo from the kitchen, if he so wanted. When I asked him why he'd want to do that he knew no answer, and when I asked him what the system cost, I was aghast. As I said, that was twenty years ago, but since then I associated switching lights with home automation.

In today's Internet-connected world there are an ever-growing number of nice gadgets (some of them are soo sweet -- it's hard to resist, but I tend to) that control something or other. From lights and power sockets over smoke detectors to bathroom scales and kettles, these things have Internet connectivity, and each of them has an individual supercool smartphone app that controls it. The result: a bunch of connected (but disconnected, if you know what I mean) "Things of the Internet" and a large number of Apps, one for each "Thing". (Which app do I need to switch on the kettle? Oh, no: that's the scales app, ...)

Home automation to me is more than just switching lights. Home automation is switching lights, controlling heating and other appliances, and having an intelligent system decide when to do what (from human input, of course). Lights, for example, should extinguish when everybody's left the house. On the other hand, certain lights should go on when it gets dark, even if nobody's at home, and the heating in the living room should power up just as I leave work, except when I'm on holiday. I should be able to override all of this (and much more) when on the road, and the system must be extensible, allowing integration of new appliances and "things".

Home automation here at Casa Mens has one absolute requirement: the WAF must be extremely high; anything else just won't cut it. It's OK for me to have to reboot/restart the odd component, but if I have to tell the "CEO" over the phone to do that, I needn't return home.

Getting an automation project past the "management" was astonishingly easy: I said we could save money on the cost of energy (without actually having calculated how much we'd save... I also kind of forgot to mention a startup investment), and the project commenced.

I spent many hours scourging the internets on information about thermostats, and got fed up. Along the way I heard of openHAB and got sidetracked in a test drive on my laptop whilst sitting in a hotel room. I didn't have hardware I could control, but a couple of openHAB's "bindings" (HTTP and MQTT in particular) allowed me to get a first impression of what the system was capable of. Returning home, I hooked up our Fritz!Box and the Sonos, and it all worked.

So now it was time to re-investigate hardware which could be controlled with openHAB. To cut a very long story short, I decided to ignore automating the heating (for the moment), and I settled on ELV MAX! as an entry-level system: their radiator thermostats are practical and good-looking, and the startup price was reasonable. I kept in mind that openHAB would be able to drive MAX! if I so wanted.

The MAX! system was set up and ready to roll in half an hour, and I was congratulated the following week when the bathroom was crispy warm in the morning (Winter is a good season to implement heating automation.) This would have been the end of the story, but I couldn't stop thinking about some neat things we could do with openHAB...

openHAB

openHAB controls "things", and it can talk to the outside world: it sends e-mail, speaks XMPP, launches programs, stores state in SQL (or other) databases, but it does only what you want it to; you don't need a database if you don't want it, and you don't have to invest the EUR 10,000 in a special home-automation thing if you can't. (Other systems come to mind: Fhem, the thing system, and many more, and no: I cannot compare them.)

openHAB can do things like execute a program when a particular event is detected (press a switch on the UI, fire up a UNIX program), send an e-mail or even use Twitter for notifications. I've decided to use MQTT as my notification "bus" of choice (reasons below).

If you want to mute your Sonos when the phone rings on the Fritz!Box, openHAB can accomplish that.

openHAB runs on small hardware (yes, also a Raspberry Pi) which means you don't need an expensive, power-hungry server to run it on.

Z-Wave

That's all fine and dandy, but how do I control appliances such as lights, etc.? I spent a bit of time investigating home automation networks of which there are probably dozens. Names such ashomematic, Z-Wave, KNX, are readily available, and they all have all sorts of actuators and sensors. They also have one thing in common: none of those are particularly inexpensive.

Ben Jones convinced me to look into Z-Wave, and I took the plunge and started very modestly with an Aeon Z-Wave USB stick and three Fibaro wall-socket plugs. Ben warned me that Z-Wave is addictive, and Jakob Schlyter said I was embarking on a potentially expensive journey. Both gentlemen are right. :-)

The USB thing is the Z-Wave controller which, in disconnected state, is used to include the Z-Wave nodes with the controller. Once plugged in to a USB host port, it hopefully starts chatting to the nodes which, together, create a mesh network. A Z-Wave controller can, say, tell a wall plug to switch itself on, and it receives feedback from the node whether the operation was completed.

Z-Wave consists of controllers and slaves. Slaves are sensors which transmit metered data or they are actuators which are capable of executing an action, such as turning a lamp on. To complicate the issue, some actuators contain sensors (for example, the Fibaro plugs report current consumption of the plug as well as being able to switch it on or off). Controllers (sometimes called gateways) are typically mains-powered. Further, we have battery-operated remote controllers. All this results in a large number of possible communication patterns, not all of which are supported by all devices! Basically

• • Controllers control actuators

  • Actuators report change of status back to the controller. In other words, switching, say, a lamp, will cause that switch to report back whether it actually switched -- it reports a state.

  • Sensors control actuators and actuators can control other actuators.

  • Remote controls send signals to actuators or controllers.

Still with me?

So, after associating ("including") all nodes onto the Aeon USB stick I was a bit stuck: Z-Wave nodes have numbers, but which is which?

I used DomotiGa with its Z-Wave Commander to see what my small Z-Wave network consits of:

Once I knew that the Z-Wave stuff was working and nodes were seeing each other, I wired up an initial openHAB configuration for the three wall plugs. openHAB immediately saw the plugs and I could play blinkenlights.

openHAB consists of a headless program (called the 'runtime') which provides an event bus. This receives events and sets states and timers from "items". An item is something connected to that bus; it can be a toaster, the temperature in a particular room, a light switch, an MQTT topic, or an SNMP event. Items states are obtained from an ever-increasing number of "bindings" provided by the project and I can surface individual switches from those bindings onto a UI if I so desire.

Configuring openHAB

openhab.cfg contains what I call base-level configuration of the bindings. This is a snippet from myopenhab.cfg in which I configure how the sonos binding should find our Sonos system, how the mqttbinding should talk to my MQTT broker, where my Fritz!Box is, and where the Z-Wave port is.

(While I'm at it: I'll be using a bit of MQTT in this account, but it's not a requirement: openHABdoesn't require MQTT -- it's a possible transport which I'm using because it fascinates me.)

openHAB's item and sitemap configuration is stored in plain text files which I can edit with a text editor or via a tool called the Designer, which does syntax checks etc. The runtime detects changes to the configuration and reloads them on the fly. (openHAB has Dropbox support, so I can also change configuration when en-route and have that reloaded as soon as I synchronize, but a Samba share also does the trick.) I quickly tired of the Designer and just use a text editor: openHAB reports syntax errors when it loads its configuration, and that's fine for me.

This is the configuration for a so-called item which "reads" from the MQTT binding. It translates to a number. The syntax is a bit strange-looking, but one does get used to it after a while.

It obtains the current energy consumption from my YouLess meter which I periodically publish to an MQTT topic.

The Fibaro wall plug, is an example of a Z-Wave actuator which has a bit more complex configuration with a switch and two numbers; the former switches the power proper, and the latter return current utilization and total energy consumption on the device.

So far we've seen a few item configurations. These I assemble into a so-called sitemap which looks like this and which renders to the Web interface shown up top:

Rules

openHAB has a very powerful rules system with which people do all sorts of stuff such as querying if it's dark (sensor) and if it's summer (weather service over XML) switch the lights off in the garden if the owner isn't there (presence).

My trivial example switches the kitchen light off at 22:15.

Moar Z-Wave

Remember Ben & Jakob saying "addictive" and "expensive" respectively. Damn right! I'd read about a KFOB (I still don't really know what that abbreviation means), a small remote controller on a key chainand considered that ideal to increase the WAF; it wouldn't be necessary to fire up an app on an iPad to switch lights.

This battery-operated device was a nightmare for me. I wanted it to talk to openHAB so that I could configure the latter to react to a key press on the former. Nothing doing. I gave up and re-read the light instruction manual for the 9th time to set up associations. An association (which isn't the same as an inclusion) means I can configure particular Z-Wave devices to talk directly to actuations. So, after clicking a lot of buttons, the KFOB thing told a Fibaro wall switch directly to turn itself on or off, without openHAB's intervention. (I understand this has been solved meanwhile in the Z-Wave binding, but I don't care to test it at this point.)

So, how do I control in-wall switches? Fibaro has micro-switches (called relays) which are built in to the switch directly in the wall. The switch, which doesn't have to be a toggle, any will do) tells the relay to switch, and the relay then controls the load.

They're brilliant, and they worked immediately, just like the Fibaro wall plugs. Lovely kit!

If I thought the KFOB was a nightmare, I wasn't prepared for the Aeotec MultiSensor 4-in-1.

A really nice device, theoretically anyway, but I did loose a lot of hair and sleep (full story).

The sensor is able to report four different values, and we account for that in our configuration (here just two): the Z-Wave device I use remains the same (node 10), but we use different sensor types:

The end result, all on a single sitemap looks like this, and believe me, was I happy to finally see this data being reported by the device.

Persistence

Persistence support in openHAB allows me to store item states over time using a data store (of which there are several). I can configure which items to persist (on an item- or group-basis) and how often values should be persisted.

Using HABmin (a sub-project of openHAB, hopefully soon to be integrated) I can view graphs from the persistence databases, produce tables, etc.

MQTT in openHAB

You may be wondering about my liberal use of MQTT in openHAB. I'll admit to being addicted toMQTT, and this "twitter for my network" (self-quote) allows me to easily talk to or listen to openHAB without bothering with specific binding configurations of the latter.

• 1. Alerts from openHAB are posted to MQTT, and I can notify external parties.

  2. openHAB has a Twitter action, but I already have that using MQTT, so I don't have to worry about specific Twitter configuration for openHAB.

  3. I can use MQTT inbound messages in openHAB to update items (i.e. flick switches).

The transport for the MQTT binding is configured as above in openhab.cfg, but I can also add what is called an Event Bus configuration.

this causes all commands and states to be published to an MQTT broker. (Additionally, I can configureopenHAB to receive status updates from a broker e.g. for linking two openHAB instances together.)

With above snippet, whenever openHAB sees something happening on its event bus, it will publish a message to my broker:

If you look at the last three lines, the following happened:

• 1. I switched on the kitchen lamp (command)

  2. The Fibaro switch reported its current state

  3. The Fibaro switch reported a value for power consumption

(I haven't tried yet, but my use of MQTT ought to enable a different interface to the hardware (via openHAB and MQTT) using homA.)

REST API

openHAB REST API allows me to access its sitemaps, and it allows me read access to items and theirstates as well as submitting commands to items. For example, the following item subscribes to anMQTT topic, and I would presumably display the resulting string on a sitemap (e.g. as an information display)

If I now publish a value onto that openHAB item (mosquitto_pub -t hab/mystr -m "Hello MQTT!") I could use the HTTP REST API to read it back:

or in JSON by appending ?type=json to the query

(I have formatted the output for clarity.)

I can also query the state which is returned as a plain string

and I can update that state:

or send a command to, say, switch the kitchen lamp off

Mobile

openHAB has provision for securing access to it with TLS and basic user/password authentication, and it can be configured to insist on both for a particular interface. Coupled with one of the mobile apps (for iOS and Android) this enables people to control their home from en-route.

The mobile apps display one or more of the sitemaps configured in openHAB which is good: I create a sitemap which is can be controlled by a Web browser or the mobile apps.

I personally don't want to be able to control my home from the Internet (I'm a little paranoid, I admit), but many people want to be able to do that.

Wrapping up

openHAB isn't easy to learn (and I'm far from understanding all that is going on under the hood), but it certainly is very powerful. I'll admit to groaning at the fact that it's pure Java, but it really does run on a small platform. (Even so, I recommend something faster than a Raspberry Pi, at least at the beginning when you will be stopping/starting openHAB frequently, because startup times on the Pi are abysmal.) openHAB's documentation is sufficient, but the really good configuration examples are strewn over the mailing-list; that's where to get help -- there are some really experienced people who are willing to help.

I'm very pleased with the hardware; Z-Wave works well with good devices, but you need a bit of patience to get it all going smoothly. Once everything is correctly set up, it seems to just work. (Remind me about the battery-powered MultiSensor to get my blood boiling...) The kit isn't cheap, but I think it's there to last; even if I decide to no longer use openHAB (which I doubt at this time), there's plenty of software support for Z-Wave devices.

Related:

• Who phoned? FritzBox, openHAB, and MQTT know

• On the failure to monitor our openHAB installation