Using Azure IoT Hub to connect my home to the cloud

I’ve written about my hybrid local/cloud home automation architecture previously: in summary most of the moving parts and automation logic live on a series of Raspberry Pis on my home network, using a MQTT broker to communicate with each other. I bridge this “on-prem” system with the cloud in order to route incoming events, e.g. actions initiated via my Alexa Skills, from the cloud to my home, and to send outgoing events, e.g. notifications via Twilio or Twitter.

Historically this bridging was done using Octoblu, having a custom Octoblu MQTT connector running locally and an Octoblu flow running in the cloud for the various inbound and outbound event routing and actions. However, with Octoblu going away as a managed service hosted by Citrix, I, like other Octoblu users, needed to find an alternative solution. I decided to give Azure IoT Hub and its related services a try, partly to solve my immediate need and partly to get some experience with that service. Azure IoT isn’t really the kind of end-user/maker platform that Octoblu is, and there are some differences in concepts and architecture, however for my relatively simple use-case it was fairly straightforward to make Azure IoT Hub and Azure Functions do what I need them to do. Here’s how.

I started by creating an instance of an Azure IoT Hub, using the free tier (which allows up to about 8k messages per day), and within this manually creating a single device to represent my entire home environment (this is the same model I used with Octoblu).

After some experimentation I settled on using the Azure IoT Edge framework (V1, not the more recently released V2) to communicate with the IoT Hub. This framework is a renaming and evolution of the Azure IoT Gateway SDK and allows one or more devices to be connected via a single client service framework. It is possible to create standalone connectors to talk to IoT Hub in a similar manner to how Octoblu connectors work, but I decided to use the Edge framework to give me more flexibility in the future.

There are various ways to consume the IoT Edge/gateway framework; I chose to use the NPM packaged version, adding my own module and configuration. In this post I’ll refer to my instance of the framework as the “gateway”. The overall concept for the framework is that a number of modules can be linked together, with each module acting as either a message source, sink, or both. The set of modules and linkage are defined in a JSON configuration file. The modules typically include one or more use-case specific modules, e.g. to communicate with a physical device; a module to bidirectionally communicate with the Azure IoT Hub; and a mapping module to map between physical device identifiers and IoT Hub deviceNames and deviceKeys.

The requirements for my gateway were simple:

  1. Connect to the local MQTT broker, subscribe to a small number of MQTT topics and forward messages on them to Azure IoT Hub.
  2. Receive messages from Azure IoT Hub and publish them to the local MQTT broker.

To implement this I built a MQTT module for the Azure IoT Edge framework. I opted to forego the usual mapping module (it wouldn’t add value here) and instead have the MQTT module set the deviceName and deviceKey for IoT Hub directly, and perform its own inbound filtering. The configuration for the module pipeline is therefore very simple: messages from the IoT Hub module go to the MQTT module, and vice-versa.

The IoT Edge framework runs the node.js MQTT module in an in-process JavaScript interpreter, with the IoT Hub being a native code module that runs in the same process. Thus the whole gateway is run as a single program with the configuration supplied as its argument.

The gateway runs on a Pi with my specific deviceName and deviceKey, along with MQTT config, stored locally in a file “/home/pi/.iothub.json” that look like this:

    "deviceKey":"<deviceKey for device as defined in Azure IoT Hub>",
    "protocol":"{\"protocolId\": \"MQIsdp\", \"protocolVersion\": 3}"

The gateway can now happily send and receive messages from Azure IoT Hub but that isn’t very useful on its own. The next step was to setup inbound message routing from my Alexa Skills.

aziotIn the previous Octoblu implementation the Alexa Skills simply called an Octoblu Trigger (in effect a webhook) with a message containing a MQTT topic and message body. The Octoblu flow then sent this to the device representing my home environment and the connector running on a Pi picked it up and published it into the local MQTT broker. The Azure solution is essentially the same. I created an Azure Function (equivalent to an AWS Lambda function) using a JavaScript HTTP trigger template, that can be called with a topic and message body, this then calls the Azure IoT Hub (via a NPM library) to send a “cloud-to-device” (C2D) message to the MQTT gateway device – the gateway described above then picks this up and publishes it via the local broker just like the Octoblu connector did. I then updated my Alexa Skills’ Lambda Functions to POST to this Azure Function rather than to the Octoblu Trigger.

The code for the Azure function is really just argument checking and plumbing to call into the library that in turn calls the Azure IoT Hub APIs. In order to get the necessary Node libraries into the function I defined a package.json and used the debug console to run “npm install” to populate the image (yeah, this isn’t pretty, I know) – see the docs for details on how to do this.

If you’re wondering why I’m using both AWS Lambda and Azure Functions the reason is that Alexa Smart Home skills (the ones that let you do “Alexa, turn on the kitchen lights”) can only use Lambda functions as backends, they cannot use general HTTPS endpoints like custom skills can. In a different project I have completely replaced an Alexa Skill’s Lambda function with an Azure function (which, like here, calls into IoT Hub) to reduce the number of moving parts.

So with all of this I can now control lights, TV, etc. via Alexa like I could previously, but now using Azure IoT rather than Octoblu to provide the cloud->on-prem message routing.

logicappThe final use-case to solve was the outbound message case, which was limited to sending alerts via Twitter (I had used Twilio before but stopped this some time back). My solution started with a simple Azure Logic App which is triggered by a HTTP request and then feeds into a “Post a Tweet” action. The Twitter “connection” for the logic app is created in a very similar manner to how it was done by Octoblu, requiring me to authenticate to Twitter and grant permission for the Logic App to access my account. I defined a message scheme for the HTTP request which allowed me to POST JSON messages to it and use the parsed fields (actually just the “message” field for now) in the tweet.

I then created a second Azure Function which is configured to be triggered by Event Hub messages using the embedded Event Hub in the IoT Hub (if that all sounds a bit complex, just use the function template and it’ll become clearer). In summary this function gets called for each device-to-cloud event (or batch of events) received by the IoT Hub. If the message has a topic of “Alert” then the body of the message is sent to the Logic App via its trigger URL (copy and paste from the Logic App designer UI). I added the “request” NPM module to the Function image using the same procedure as for the iot-hub libraries above.

The overall flow is thus:

  1. Something within my home environment publishes a message on the “Alert” topic.
  2. The Azure IoT gateway’s MQTT module is subscribed to the “Alert” topic, receives the published message, attaches the Azure IoT Hub deviceName and deviceKey and sends it as an event via the IoT Hub module which sends it via AMQP to Azure IoT Hub.
  3. Azure IoT Hub invokes the second Azure Function with the event.
  4. The Function pulls out the MQTT topic and payload from the event and calls the Logic App with them.
  5. The Logic App pulls out the message payload and send this as a tweet using the pre-authorised Twitter connector.

Although all of this seems quite complex, it’s actually fairly simple overall: the IoT hub acts an a point of connection, with the on-prem gateway forwarding events to and from it, and a pair of Azure Functions being used for device-to-cloud and cloud-to-device messages respectively.


It was all going well until I discovered that the spin-up time for an Azure Function that’s been dormant for a while can be huge – well beyond the timeout of an Alexa Skill. This is partly caused by the time it takes for the function runtime to load in all the node modules from the slow backing store, and partly just slow spin-up of the (container?) environment that Azure Functions run within. A common practice is to ensure that functions are invoked sufficiently often that Azure doesn’t terminate them. I followed this practice by adapting my existing heartbeat service running on a Pi that publishes a heartbeat MQTT every 2 minutes to also have it call the first Azure function (the one that the Alexa Skills call) with a null argument; and to keep the second function alive I simply had the MQTT gateway subscribe the heartbeat topic thereby ensuring the event handler function ran at least once every 2 minutes as well.



A universal IR remote control using MQTT

In some previous hacking I created an add-on for the Kodi media player which allowed me to control Kodi and the TV it is connected to (by using an IR blaster) using messages published through my home MQTT broker. The original purpose for this hack was to enable voice control via an Amazon Echo Smart Home Skill.

alloffI’ve since added another use-case where a single push-button connected to a Raspberry Pi Zero W publishes a single MQTT message which my rules engine picks up and then publishes a number of messages to act as an “all off” button: it sends “off” messages to all lights in the room (these are a mixture of LightwaveRF and Philips Hue – both interfaced via services that subscribe to the local MQTT broker); a “pause” message to Kodi; and a TV “off” message to the TV.

However, despite having this capability I still use separate traditional IR remote controls for the TV and Kodi, and a 433MHz control for the LightwaveRF lights. It seemed like a good idea to take advantage of the MQTT control to reduce the need for so many remote controls so I set about turning the Hama remote control I use for Kodi into a universal control for all the devices.

The strategy I used was to have the Hama remote control publish MQTT messages and add some rules to the broker’s rules engine to map these to the required MQTT messages(s) to control Kodi, the TV, or the lights. I chose to connect the Hama USB IR receiver to a new Raspberry Pi Zero W – I could have left this connected to the Pi 3 running Kodi and created a new add-on to talk to it but I have future plans that call for another Pi in this location and this seemed like it would be easier… – and set about building a small service to run on the Pi to relay received IR commands to the MQTT broker.

Overview of the overall architecture

After a few false starts with LIRC I settled on consuming events from the remote control via the Linux event subsystem (same as handles keyboard and mouse). There are some node.js libraries to enable this but I found a much more complete library available for Python which, critically, implements the “grab” functionality to prevent “keystrokes” from the IR control also going to the Pi’s login console.

I’ve already implemented a few Python MQTT clients using the Paho library (including the Kodi add-on itself) so I recycled existing code and simply added an input listener to attach to the two event devices associated with the IR control (hard-coded for now) and, after a little processing of the event, publish an MQTT message for each button press. The Hama remote acts like a keyboard and some of the buttons include key modifiers: this means that a single button push could involve up to 6 events: e.g. key-down for left-shift, key-down for left-ctrl, key-down for ‘T’, followed by three key-up events in the reverse order. My code maintains a simple cache of the current state of the modifier keys so that when I get a key-down event for a primary key (e.g. ‘T’ in the above example) I can publish a MQTT message including the key and its active modifiers.

 for event in self.device.read_loop():
     if event.type == evdev.ecodes.EV_KEY:
         k = evdev.categorize(event)
         set_modifier(k.keycode, k.keystate)
         if not is_modifier(k.keycode) and not is_ignore(k.keycode):
             if k.keystate == 1:
                 msg = k.keycode + get_modifiers()
                 self.mqttclient.publish(self.topic, msg)

(The full code for the service can be found here.)

This results in MQTT messages of the form

IR/room2av KEY_LEFT
IR/room2av KEY_RIGHT
IR/room2av KEY_DOWN
IR/room2av KEY_UP

The next step was to add a rule to the rules engine to handle these. The rules engine is a simple MQTT client that runs on the same Raspberry Pi as the MQTT broker; it listens to topics of interest and based on incoming messages and any relevant state (stored in Redis) publishes message(s) and updates state. In this case there is no state to worry about, it is simply a case of mapping incoming “IR/*” messages to outbound messages.

A (partial) example is:

function handle(topic, message, resources) {
  switch (topic) {
  case "IR/room2av":
    switch (message.toString()) {
    case "KEY_UP":
      resources.mqtt.publish("KODI/room2/KODI", "Action(Up)");
    case "KEY_DOWN":
      resources.mqtt.publish("KODI/room2/KODI", "Action(Down)");
    case "KEY_PAGEUP":
      resources.mqtt.publish("Light/room2/Lamp", "on");
      resources.mqtt.publish("Light/room2Ceiling", "on");
    case "KEY_PAGEDOWN":
      resources.mqtt.publish("Light/room2/Lamp", "off");
      resources.mqtt.publish("Light/room2Ceiling", "off");
    case "KEY_VOLUMEUP":
      resources.mqtt.publish("KODI/room2/TV", "VOL_p");
    case "KEY_VOLUMEDOWN":
      resources.mqtt.publish("KODI/room2/TV", "VOL+m");

Here we can see how buttons pushes from this one IR remote are routed to multiple devices:

  • the “up” and “down” navigation buttons result in messages being sent to Kodi (the message content is simply passed to Kodi as a “builtin” command via the xbmc.executebuiltin(…) API available to add-ons);
  • the “+” and “-” channel buttons (which map to PAGEUP and PAGEDOWN keycodes) have been abused to turn the lights on and off – note the two separate messages being sent, these actually end up going to LightwaveRF and Philips Hue devices respectively; and
  • the “+” and “-” volume buttons send IR commands to the TV (this happens to be via the Kodi add-on but is distinct from the Kodi control) – the “VOL_p” and “VOL+m” being the names of the IR codes in the TV’s LIRC config file.

A major gotcha here is that when controlling a device such as the TV with an IR blaster, there will be an overlap between the blast of IR from the Hama device and from the IR blaster connected to the Kodi Pi, and the TV will find it difficult to isolate the IR intended for it. To avoid this I’ve had to put tape over the TV’s IR receiver and IR blaster which is glued to it such that IR from the Hama control can’t get through.

The end result is that I can now use a single IR remote control to navigate and control Kodi, turn the TV on and off and adjust its volume, and control the lights in the room. Because everything is MQTT under the hood, and I’ve got plumbing to route messages pretty much anywhere I want, there is no reason why that IR remote control can’t do other things too. For example it could turn off all the lights in the entire house, or turn off a TV in another room (e.g. if I’ve forgotten to do so when I left that room), or even to cause an action externally via my Azure IoT gateway (more on this in a future blog post). And because the rules engine can use state and other inputs to decide what to do, the action of the IR remote control could even be “contextual”, doing different things depending on circumstances.


Automating HDMI input switching

I recently wrote about automating TV power control using IR LEDs. This enabled me to turn a TV and amplifier on and off with Amazon Echo and as part of other automation flows. However, with a cable TV box, Kodi media player, PlayStation 3 and Google Cast I still needed to choose the TV’s input source manually. I’d like to be able to say “Alexa, turn on cable”, or “Alexa, turn on Google Cast” and have it not only turn on the TV and amp as it can now, but also switch to the correct input.

One option would be to take advantage of automatic HDMI switching driven from a source coming online – but with things like the Cast and Kodi player being always-on this made this a no-go.

Another option was to use an external HDMI switcher box with an IR remote control and blast IR commands at this in the same way as for the TV itself. In fact this was my original plan and I carefully selected a HDMI switcher that was not only getting decent reviews, but also had a remote control button per channel (simply having a “next input” cycle button was not suitable because I’d have no way to know when I got to the input I wanted). However when I received the switcher I thought I’d try something different, just for fun.


This particular switcher has an LED for each input, showing which is selected, and a single push button to cycle through the inputs. Having opened up the box by prising off the plastic feet and unscrewing the 4 small screws underneath I took a good look at the circuit board. I observed that the LEDs were wired with the cathodes connected to ground, therefore the anodes would be at a positive voltage when the LED was lit – this is good enough to directly drive a Raspberry Pi GPIO input pin in pull-up mode. By connecting all 5 LED anodes to GPIO pins I could easily tell which input was selected. The push button was also connected to ground, suggesting it was being used in a pull-up fashion. I took a gamble that I could get away with driving this directly from a Raspberry Pi GPIO output without any further interfacing. To make life even easier all of the LED and switch pins were through-hole soldered into the switcher’s PCB (i.e. not surface mount like most of the board) meaning small wires could be fairly easily soldered onto them.


Even more conveniently the arrangement of the box, although very compact, had plenty of room to bring in a 9 core cable (5 LEDs, 1 switch, 1 ground and two unused wires) leading to a neat appearance. To route the cable out I drilled a 5mm hole close to the IR socket on the edge of the box and routed the wires over the top side of the PCB.

On the Raspberry Pi end I simply chose 6 unused GPIO pins and connected the LEDs and switch input as well as connecting ground on the HDMI box to a ground pin on the GPIO header.


I wanted to use the same Raspberry Pi running LibreELEC and Kodi that I was already using to drive the IR emitters. This meant extending the existing Kodi add-on with the capability to read the HDMI switcher’s LED status and drive its push button. The general strategy was to have an incoming MQTT message set the desired input number (1-5) and then send button push pulses until the associated LED lit up. One potential gotcha here is that this switcher skips over inputs that do not have an active HDMI source, therefore if the MQTT message requests an input with either nothing connected, or with a source that’s not switched on, the button pushing could go on forever. To avoid this I limited the number of button push attempts per MQTT message to 10.

The prerequisite was to add the “Script – Raspberry Pi Tools” Kodi add-on (search the registry from the Kodi add-on manager UI) to add the GPIO Python library.

The code is pretty simple. At start of day I set up GPIO – the first 5 pins are the inputs from the switcher’s LEDs and the final pin is the output to the push button:

import RPi.GPIO as GPIO

# Setup pins for HDMI switcher control
GPIO.setup(21, GPIO.IN, pull_up_down = GPIO.PUD_UP) 
GPIO.setup(8, GPIO.IN, pull_up_down = GPIO.PUD_UP) 
GPIO.setup(16, GPIO.IN, pull_up_down = GPIO.PUD_UP) 
GPIO.setup(12, GPIO.IN, pull_up_down = GPIO.PUD_UP) 
GPIO.setup(7, GPIO.IN, pull_up_down = GPIO.PUD_UP)
GPIO.setup(20, GPIO.OUT)
GPIO.output(20, 1)

And initialise some global variables to record the desired input number and the count of button push attempts:

hdmiinput = 1
hdmiattempts = -1

In the MQTT message handler I added another sub-topic case:

            elif ll[2] == "HDMI":
                    hdmiinput = int(msg.payload)
                    hdmiattempts = 0

And in the main loop (which cycles every 0.5s):

        if (hdmiattempts > -1) and (hdmiattempts < 10):
            for i in range(4):
                hdminow = 0
                if GPIO.input(21): hdminow = 1
                if GPIO.input(8): hdminow = 2
                if GPIO.input(16): hdminow = 3
                if GPIO.input(12): hdminow = 4
                if GPIO.input(7): hdminow = 5
                log("HDMI switcher currently on %s, want %s" % (hdminow, hdmiinput))
                if hdminow == hdmiinput:
                    hdmiattempts = -1
                    log("Sending HDMI switch button trigger")
                    GPIO.output(20, 0)
                    GPIO.output(20, 1)
                    hdmiattempts = hdmiattempts + 1

The full Kodi add-on code can be found here. It’s quick and dirty, but it works.

Automation with Echo

So now I have a way to change HDMI input to the TV by sending a MQTT message such as “KODI/Lounge/HDMI=4“. To use this with Amazon Echo I extended my existing TV control with specific cases for each of the 4 inputs in use. The configuration for the Alexa smart home skill adapter (see Controlling custom lighting with Amazon Echo and a skill adapter for more on this) sends a single MQTT command into my home broker:

                    "manufacturerName":"James Bulpin",
                    "friendlyName":"Virgin Media",
                    "friendlyDescription":"Living room TV and amp on cable TV",
                    "manufacturerName":"James Bulpin",
                    "friendlyName":"Chrome Cast",
                    "friendlyDescription":"Living room TV and amp on Google Cast",
                    "manufacturerName":"James Bulpin",
                    "friendlyDescription":"Living room TV and amp on PS3",
                    "manufacturerName":"James Bulpin",
                    "friendlyDescription":"Living room TV and amp on Kodi",

(See this gist for the context – these rules are just added to the list)

Rules in the rules engine then turn this into TV power control and in the “on” case, also sends the HDMI control message:

  case 'compound/loungetvcable':
    resources.mqtt.publish("KODI/Lounge/TV", message.toString());
    if (message.toString() == "on") {
      resources.mqtt.publish("KODI/Lounge/HDMI", "1");
  case 'compound/loungetvcast':
    resources.mqtt.publish("KODI/Lounge/TV", message.toString());
    if (message.toString() == "on") {
      resources.mqtt.publish("KODI/Lounge/HDMI", "2");
  case 'compound/loungetvps3':
    resources.mqtt.publish("KODI/Lounge/TV", message.toString());
    if (message.toString() == "on") {
      resources.mqtt.publish("KODI/Lounge/HDMI", "5");
  case 'compound/loungetvkodi':
    resources.mqtt.publish("KODI/Lounge/TV", message.toString());
    if (message.toString() == "on") {
      resources.mqtt.publish("KODI/Lounge/HDMI", "4");

In closing

By adding explicit input selection to the set of MQTT commands I can send I can now build more complex automation actions that not only turn things on or off but can also select inputs. This enables a single Echo command such as “Alexa, turn on Chrome Cast” to get all the necessary devices into the right state to achieve the desired outcome of the TV and amp being on and displaying the output of the Cast.



Using infrared control from Amazon Echo

A while back I built a skill adapter to allow Amazon Echo’s smart home skill system to manage my LightwaveRF lights via my custom Octoblu+MQTT system. But why stop there? If I can say “Alexa, turn on the living room light” why can’t I also say “Alexa, turn on the TV“? With IoT mains power sockets such as Belkin Wemo or LightwaveRF I could power on and off the TV but I prefer to leave the TV in standby (yes, I know this uses power and kills the planet). Instead I decided to solve the problem by using infrared emitters glued to the front of the devices I wanted to control and allowing Alexa to control these.


ircircuit2Each TV in my house is co-located with a Raspberry Pi running the Kodi media player. Therefore it made sense to add the IR emitter capability to these rather than add more hardware. There are various options for emitting IR including USB IR blasters and professional systems design for commercial AV use-cases. I chose a homebrew route and built my own emitters using an IR LED, a NPN transistor and a couple of resistors. The whole thing connects to a Raspberry Pi GPIO pin. Using a transistor allows for higher LED currents than can be used directly from the GPIO pin and allows multiple emitters to be connected to the same GPIO pin, e.g. to connect several home entertainment devices.


I wired the four components together in free space (no board) and connected to a three core cable. I used offcuts of cable sheath and heatshrink tubing to insulate and encapsulate everything. The end result is an IR LED poking out the top of black blob on the end of a black cable. It looks a bit ugly close-up but once attached to a black TV case it’s hardly noticeable.

I connected the other end of the cable to the GPIO header on the Raspberry Pi using +5V, GND and GPIO17 (this is the default for the LIRC Pi GPIO drive) pins. I then positioned and superglued the emitter onto the front of the TV such that it could see the TV’s IR receiver but without completely blocking it.


Kodi interface

The Raspberry Pi runs the LibreELEC distribution of Kodi. To set this up to use GPIO-driven IR required me to SSH to the Pi as root and edit /flash/config.txt to add a line “dtoverlay=lirc-rpi” – this is needed to load the kernel driver for Raspberry Pi GPIO for the Linux IR subsystem. I then needed to download the specific IR configurations I needed for my devices. For example for my Sony Bravia TV I ran:

 wget -O "/storage/.config/lircd.conf" \

For controlling multiple devices multiple remote control configs can be concatenated into the lircd.conf file.

After a reboot it was time to code a Kodi add-on. This add-on runs as a service and connects to my home MQTT broker in order to receive published messages (more of which later). The MQTT message handler spawns the “irsend” command (comes with the LibreELEC distribution) to send the named IR command(s). For example, the following code is used to control the TV and an amplifier at the same time (using two IR emitters wired to the same GPIO pin):

# The callback for when a PUBLISH message is received from the server.
def on_message(client, userdata, msg):
    global player, hdmiattempts, hdmiinput

        msgpayload = str(msg.payload)
        print(msg.topic+" "+msgpayload)
        ll = msg.topic.split("/")
        if len(ll) > 2:
            if ll[2] == "TV":
                log("Incoming TV command: " + msgpayload)
                ircmds = []
                if msgpayload.lower() == "off":
                    ircmds.append(["Sony_RM-ED009-12", "KEY_POWER", 5])
                    ircmds.append(["Marantz_RMC-73", "Amp_Standby", 1])
                elif msgpayload.lower() == "on":
                    ircmds.append(["Sony_RM-ED009-12", "KEY_POWER", 5])
                    ircmds.append(["Marantz_RMC-73", "Amp_Standby", 1])
                    ll = msgpayload.split("/")
                if len(ircmds) > 0:
                    log("Sending IR commands: " + str(ircmds))
                    for ircmd in ircmds:
                        cmd = "/usr/bin/irsend --count %u -d /run/lirc/lircd-lirc0 SEND_ONCE %s %s" % (ircmd[2], ircmd[0], ircmd[1])
    except Exception, e:
        log("MQTT handler exception: " + str(e))

I developed the add-on in-place on the Raspberry Pi (as opposed to building a distributable package and importing it through Kodi’s UI). To do this I firstly created a directory:

# mkdir /storage/.kodi/addons/service.mqtt.jamesbulpin

Within this I created two files:

And a subdirectory “resources” containing:

Overall the structure was this:

LoungeTV:~/.kodi/addons/service.mqtt.jamesbulpin # find . -type f

The full code can be found at – note that this includes hard-coded IR devices and commands so would need customisation before use.

kodiconfigThe easiest way to get Kodi to discover the add-on is to reboot. It may also be necessary to enable the add-on via the “My add-ons” menu in Kodi. The settings dialog can also be accessed from here – this allows setting of the MQTT broker IP address and of the name that will be used in the MQTT topic the add-on subscribes to – if this is set to “Lounge” then the topic prefix is “KODI/Lounge“.

With the add-on running I tested it by manually sending MQTT messages:

# mosquitto_pub -m on -t KODI/Lounge/TV

Control with Amazon Echo

So far I’ve described how to get from a MQTT message through to an infrared control action on a TV. The next step was to have the MQTT message be sent in response to an Amazon Echo voice command. Happily this was very easy because I’d already built infrastructure to send MQTT commands (albeit to manage lighting) from Alexa. To add TV control was just a matter of adding another entry to the list of devices returned by the skill adapter when a discovery was initiated, and then initiating a discovery (“Alexa, discover my smart home devices“).

Example of an existing entry:

                    "manufacturerName":"James Bulpin",
                    "friendlyName":"Dining room lights",
                    "friendlyDescription":"Fireplace lights in the dining room",

New entry for the TV:

                    "manufacturerName":"James Bulpin",
                    "friendlyDescription":"Living room TV and amp",

You can see this just uses the KODI MQTT message instead of a lighting message. Alexa has no idea it’s a TV rather than a light that’s being controlled. The entire skill adapter code can be seen here.

Of course it’s also possible to control multiple devices from a single Alexa command, just by sending multiple MQTT messages.

An enhancement

This all works well but you may have noticed that the IR commands to turn the TV on and off are the same – this is because that’s usually how IR remote controls work. This means that any command, via Alexa or otherwise, will actually toggle the current power state rather than move to the defined state. In most cases this isn’t a problem, for example why would you ask Alexa to turn the TV on if it’s on already? However for other more complicated automation activities I’d like to be able to control the TV without knowing whether it’s on or off already (e.g. I want an “all off” button or timer for a bedroom that will turn off all lights and the TV no matter what was already on).

To do this I added an input to sense if the TV was on – power being available on a TV USB port was my chosen method. I used a small opto-isolator, driven by the USB power, with the other side connected to a GPIO input.

To use this in the Kodi add-on I first had to add the “Script – Raspberry Pi Tools” add-on (search the registry from the Kodi add-on manager UI to find this) to add the GPIO Python library. To use this from my add-on code I added:

import RPi.GPIO as GPIO

# Set up pin for TV power monitoring (active low)
GPIO.setup(23, GPIO.IN)

Then added simple checks in the MQTT message handler to only send IR commands if the current state is not the commanded state:

                if msgpayload.lower() == "off":
                    if GPIO.input(23) == 0:
                        # Only send if the TV is on - (GPIO pin is low)
                        ircmds.append(["Vestel_TV", "OFF", 5])
                elif msgpayload.lower() == "on":
                    if GPIO.input(23) == 1:
                        # Only send if the TV is off - (GPIO pin is high)
                        ircmds.append(["Vestel_TV", "OFF", 5])

In a future blog post I’ll show how I extended this TV control to add automatic input source selection by interfacing to, and automating a HDMI switcher box.


Hybrid local/cloud style home automation

When I first started automating items in my home I used a local (“on-prem” if you like) point solution for security lighting. This had a connection to an internet service (Google Calendar) but only in a “pull” manner, and only for that specific use-case. As time went on I wanted to enable a broader set of use-cases, some of which required other internet-connected services in both pull and push directions. I experimented with using Octoblu, with the local LightwaveRF controller being connected to it – all the automation therefore was being performed in the cloud. As I looked at adding additional devices (PIR detectors, reed switches, IR blasters, and so on) I encountered a number of failures and delays caused by this reliance on my domestic internet connection and the (at the time) patchy reliability of Octoblu (it’s far more solid now).

My solution was to use a “hybrid” model with a local message broker and set of services running inside the home, and a connection to Octoblu for cases when external services were involved. Overall this means that truly local operations, such as PIRs turning on lights depending on the position of various reed switches, can be kept within my home network with no reliance on my internet connection or any external service. In cases where external stimuli are used (e.g. for connecting Amazon Echo’s smart home stuff) or external actions are required (e.g. sending a text message using Twilio) the bridge between the two worlds allows a message to be forwarded in either direction.

The on-prem piece

homeautopiThe local system is based on a MQTT broker running on a Raspberry Pi. This provides a publish-subscribe messaging system that various services connect to. The broker and many of the services run on the same Raspberry Pi however the broker also listens on a TCP port to allow other Raspberry Pis (and potentially other devices) in the home to join the party. I also have a Redis key-value-pair (KVP) store which is used to store state such as presence information or whether it’s day or night.

As an illustration of how the system is used: the LightwaveRF radio-controlled lighting system is connected via a small node.js service that subscribes to lighting messages (e.g. “Light/Kitchen/Main=on“) and sends the appropriate command to the local LightwaveRF hub which then emits the 433MHz radio command. When I later added a Philips Hue system alongside the LightwaveRF one I created a similar node.js service that also subscribes to lighting messages and makes calls to the Hue hub’s API as necessary. In both cases the services ignore messages about lights they don’t control. This all means other services, such as the PIR detector, or the rules engine, can simply publish a lighting control message without worrying about whether it’s a LightwaveRF, Hue, or (perhaps in the future) other system managed locally or via a cloud service.

Other services that publish and/or subscribe to MQTT messages include:

  • A Bluetooth Low Energy (BLE) advertisement monitor – this is used to determine the presence of absence of various things
  • A Google calendar interface that can update local KVP state, or generate messages, based on scheduled events
  • A GPIO service that enables things like PIRs and push buttons to be attached to Raspberry Pis
  • A timer service that other services and rules can use to provide timeouts, e.g. to turn off PIR-controlled lights after a set delay
  • Plug-ins for the Kodi media player to control Kodi itself as well as IR emitters for nearby TVs and other devices (stay tuned for a blog post on this)

homeautoruleA core part of the whole system is the rules engine. This is a service that subscribes to MQTT messages, implements rules, and publishes MQTT messages as necessary. It doesn’t directly interface to any device. An example of a rule is managing lighting for the basement stairs: inputs to the rule are two PIRs (via the GPIO service), the value of the day/night KVP, and the timeout message from a named timer (from the timer service). If a PIR triggers and it is night (from the KVP) then a lighting message (“Light/Stairs/Basement=on“) is published; when the PIR stops detecting, a timer control message is published (“Timer/basement_stairs_lighting/reset=<time in 60s>“) to start the timer; receipt of the timer’s timeout message causes the rule to emit an “off” lighting message.

The Octoblu bridge


So, what if I want to control stuff in the home from somewhere else? Or if I need to send a message to a cloud-connected device or service? That’s where the Octoblu connector comes in. I built a basic connector that runs as a node.js service on the main MQTT Raspberry Pi. This subscribes to a subset of MQTT topics and has the ability to publish arbitrary messages into the on-prem system. It also acts as an Octoblu connector, maintaining a bidirectional connection into Octoblu. This allows my entire home automation world to appear as a thing in Octoblu so I can send messages to it, which leads to MQTT published messages, and receive messages from it, which came from MQTT subscribed messages.

As an example, I can remote-control a light using a button on a web page by connecting an Octoblu “trigger” (a thing that can respond to a HTTPS POST and then emit a message into the Octoblu platform) to the home automation thing, and ensure the message payload includes the right MQTT topic (e.g. “Light/Kitchen/Main“) and message (e.g. “on“). The connector routes this message into my home as a published MQTT message that the LightwaveRF or Hue service will act on. This is the core mechanism I used when connecting my home lighting up to Amazon Echo using an Alexa smart home skill adapter.

In the other direction an on-prem service, such as the rules engine, can send a message to Octoblu just by publishing to a suitable topic (currently this means it has a “Octoblu/” prefix) via the local MQTT broker. An example of this is a security feature that sends me a text message if a particular combination of stimuli are seen. The rules engine publishes a message “Octoblu/alert/<detail>=<status>“, the Octoblu connector receives this via its MQTT subscription and sends it into Octoblu. The Octoblu flow then decides how to process this message which may end up with a call to the Twilio thing to send a text message.

Other examples of using the Octoblu connection include:

  • Sending events for sunrise and sunset, based on pulling data from a weather API (saves having to drill walls to install a light sensor)
  • Routing messages from Alexa skills handlers (these run in the cloud as AWS Lambda functions so I need a way to route the message back across my firewall)

In closing

I’ve ended up with a hybrid-cloud style of IoT management for my home automation, not by design, but by evolution. I find that being able to combine local automation – with its low latency and lack of reliance on external connectivity and services, with a powerful cloud-based automation platform able to send and receive from a variety of cloud/internet services, is a best-of-both-worlds solution.


Using LightwaveRF switches to control other stuff

img_9028I’ve been using LightwaveRF‘s range of wirelessly controlled lighting and power outlets for some time and I’ve built up a collection of switches and remote controls for them. These switches broadcast control messages, formed of a command (e.g. on/off) and a unique identifier for the transmitter, using a simple on-off keying modulation on 433.92MHz with each light fitting or other device being “paired” to one or more transmitters. In reality the pairing is actually just telling the device which command(s) to respond too – the transmitter units can’t receive anything which prevents a bidirectional pairing. This all runs in parallel with the LightwaveRF network bridge which is really just an IP-controlled 433.92Mhz transmitter.

With my home automation system now growing to include Philips Hue and various homemade devices I’d like the LightwaveRF switches and remote controls to be able to control more than just LightwaveRF devices which means taking them our of their constrainted world of 433.92MHz. To do this I am building a set of 433.92MHz receivers to forward the command messages to my MQTT broker. This means I can extend my existing set of automation rules to react to the LWRF switch commands by, for example, sending a command to the Hue Hub to control a Hue light. I can also build up more complex operations with other rules such as only performing certain actions outside of daylight hours, or controlling multiple devices at the same. This also allows for more interesting control of the LightwaveRF devices themselves because I can now insert rules between the transmitter and receiver simply by not pairing the device with the transmitter and instead routing everything through the MQTT world and the LightwaveRF network bridge.

img_9021The prototype is based on three main components:

  1. A 433.92MHz receiver – I’m using a cheap one from Amazon. This has a 17.3cm wire antenna soldered on to it.
  2. An Arduino Uno R3 to handle the demodulation of the on-off keying and the protocol decoding.
  3. An existing Raspberry Pi to forward the decoded messages from the Arduino to the MQTT broker.

Why the separate Arduino? That’s because I don’t then have to worry about responsiveness of the decoding software if it was to be running on a Raspberry Pi alongside other stuff – some of which may also be latency sensitive.

There is plenty of material online (see the resources below) about the LightwaveRF protocol. I found that none of the existing code (at least what I could find) worked in the way I wanted so I implemented a decoder from scratch using the various online protocol resources as guides.The decoder, running on the Arduino, is formed of three main pieces. Firstly a demodulator that looks for state changes on the data coming from the 433.92MHz receiver. This then feeds the demodulated bits into the main protocol decoder which looks for the message start bit, each byte’s start bit, and a valid pattern for the byte (which is a form of 4b8b encoding). The final piece kicks in once the transmission has ceased for 25ms: the buffered received data is de-duplicated and for each unique message a MQTT message is sent over the serial port (via USB).

The format of the MQTT message is topic=”LWRF433/<unitID>/<channel>”, message=”<command 0=on,1=off>|<level>”, where:

  • <unitID> is the unique 3 byte identifier for the switch unit
  • <channel> is the channel number, the meaning of which varies across the different type of switches
  • <command> and <level> tell the light what to do and vary depending on whether the controlling switch is a simple switch, dimmer, or mood switch.


The initial prototype is connected to an existing Raspberry Pi running LibreELEC/Kodi as a media centre. The Arduino is connected to a USB port on this Pi. The function of the Pi is simply to forward MQTT messages received over the USB serial port from the Arduino to the house MQTT broker. To do this in the LibreELEC/Kodi environment is a little trickier than in a general Linux/Raspbian environment due to the locked-down, read-only nature of the LibreELEC distribution. However the Kodi add-on mechanism provides a reliable way to do this.

20161009_170439I created a Kodi add-on formed of two main parts. Firstly a Python script that acts as the main forwarding loop between the USB serial port and the MQTT broker. This is based on code from a previous, albeit standalone, proxy doing largely the same thing – it includes a device scanning function to watch for Arduino devices appearing and to fire up proxy threads for each one – this was originally done to avoid static configuration and to allow for hot-plugging of new Arduino devices (this was in a far more complex, multi-Arduino system, compared to this simple case).

The second part of the Kodi add-on is really just boilerplate and plumbing to kick off this script.

So putting it all together this is how it works:

  1. A button is pushed on a LightwaveRF switch, this sends a message (actually a burst of several) on 433.92MHz.
  2. The 433.92MHz receiver module receives these messages and passes the raw data to the Arduino.
  3. The software on the Arduino decodes the messages, de-duplicates and sends one or more MQTT messages over the USB serial port.
  4. The proxy script running on the Kodi Pi forwards the MQTT message(s) to the MQTT broker.
  5. Part of my MQTT rules engine is subscribed to LWRF433 messages and receives the message(s). After debouncing, the relevant rule is invoked, e.g. sending a separate MQTT message to control a Hue light.


The initial use-case is to use the LightwaveRF remote control to turn off the room lights, pause whatever the media centre is playing, and turn off the TV, all with a single button push. To do this I created a rule that on receipt of the appropriate LWRF command via MQTT sends out two more MQTT messages, one to turn off the TV and one to pause Kodi. Both of these are handled by a separate add-on running on the same Kodi Pi, but that piece of functionality is completely separate to the LWRF add-on described above and is the subject of a future blog. The room lights turn off directly in response to the LightwaveRF 433MHz command without any MQTT infrastructure being used.



Controlling custom lighting with Amazon Echo and a skill adapter

ctb2adfxeaaptil-jpg-largeAmazon launched Echo in the UK today – it’s been a long wait! My pre-ordered Echo arrived this morning and my first priority was to get it (her?) to control my home lighting. As I’ve previously written about, my LightwaveRF lights are currently managed by a custom set of scripts communicating within the house using MQTT pub-sub. This means that Amazon Echo (or Alexa, as I’ll refer to it/her from now on) doesn’t know how to interface with them like she does with Philips Hue, for example.

Luckily Amazon has made available its Smart Home Skill API which allows individuals and home automation vendors to provide “skill adapters” for the various home automation “skills” Alexa already has. This means it is possible to say “Alexa, turn on the bedroom light” and have Alexa use whatever underlying system and technology you have to execute the command. This is preferable to defining a new custom skill because it avoids the need to use a skill trigger word (e.g. “Alexa, turn on the bedroom light using LightwaveRF”). AWS Lambda is used to provide the concrete implementation for the abstract actions.

octoblualexaIn my case the skill adapter will make a HTTPS call to an Octoblu trigger node passing details of the required action (essentially just a MQTT message with the topic being the light to control and the message body being the action (on or off)). The Octoblu flow then messes about a bit with the JSON structure before passing the message to an existing Meshblu device that connects my home MQTT world with Octoblu. In reality I’m using Octoblu and Meshblu here as firewall-bridging plumbing to get a MQTT message from the Lambda function into my home environment.

Having signed up for my Amazon developer account I followed Amazon’s steps to create a Smart Home Skill. This started by creating a new skill (of type “Smart Home Skill API”) in the Alexa Skills Kit section of the Amazon Developer Console – I chose to name this skill “James’s Lights”.

To provide the actual implementation I created a Python 2.7 Lambda function named “JamesLightsSkillAdapter” in AWS (using the eu-west-1 (Ireland) region to co-locate with the Alexa voice service for the UK) based on the alexa-smart-home-skill-adapter blueprint. I based the code on the template provided in the “steps to create” page above. For the role I selected “Create new role from template(s)”.

The code handles two separate types of message from the Alexa service:

  1. Device discovery – this is an action initiated by the end user from the Alexa app (or by voice) to get an inventory of devices that Alexa can control. In the Lambda function this is implemented by returning a big blob of JSON with an entry for each device. The “friendlyName” item being the words Alexa will recognise to control the device. I’m using the additionalApplianceDetails item to record the MQTT topic that will be used to control this device. My initial prototype implementation hard-codes the entire inventory in the Lambda function – clearly not a long term solution!
  2. TurnOnRequest and TurnOffRequest commands – these are issued by the Alexa service when Alexa is asked to turn a device on or off and the device is recognised as one in the inventory. The Lambda function is called with the relevant JSON blob and therefore my code can pull out the previously configured MQTT topic and send that as part of a HTTPS POST to the Octoblu trigger mentioned above.

I tested the Lambda function using the “Alexa Smart Home – Control” sample event template, manually crafting the JSON messages to match the ones the Alexa service will be sending. After testing it’s important to make sure to enable the Alexa trigger for the Lambda function.


Back in the Developer Portal I configured the skill to use an endpoint in AWS Europe using the Lambda ARN created above. As this will be a private skill adapter I didn’t really need account linking (this is how a regular user would link their Echo with, for example, their Philips Hue account) but the console wouldn’t let me proceed without setting it up. Therefore I followed this great blog post’s instructions on how to use Amazon itself as the linked account.

file-28-09-2016-21-56-25Having saved all the skill configuration and enabling testing I then used the Alexa app on my iPad to add this newly created skill (in the “Your skills” section) and logged in with my Amazon account as the account linking step. From there is was a simple matter of letting the app and Alexa discover the devices and then Alexa was good to go.

I can now say to Alexa: “Alexa, turn the lounge light on” and a second or so later it’ll come on and Alexa will say “OK”. What happens under the hood is more interesting though:

  1. The Alexa cloud service processes the message, figuring out its a smart home control command
  2. The Alexa service looks through my discovered device list and identifies the “lounge light” is one that is controlled via this skill adapter.
  3. The Alexa service makes a call the my AWS Lambda function with a JSON message including the configuration for the requested light as well as the “TurnOnRequest” command.
  4. My Lambda function makes a HTTPS POST call to the Octoblu trigger with a MQTT-like message including the topic string for the requested light and the “on” message.
  5. The Ocotblu flow forwards this message via Meshblu to a simple Meshblu connector I have running at home.
  6. My Meshblue connector publishes the messaage to my local MQTT broker.
  7. The LightwaveRF script also running at home and subscribed to “Light/#” messages picks up the message and looks up the room/device codes which it then send via UDP to the LightwaveRF bridge.
  8. The LightwaveRF bridge sends the appropriate 433MHz transmission which is picked up by the paired light fitting and the power is switch on.

As this is all highly customized to me I’ll be leaving the app in its testing state, not putting it forward for certification (which would certainly fail!) and publication.

Future work

Right now the implementation hard-codes my device list as well as the URL of my Octoblu trigger. I’d like to, at the very least, make the device list dynamically generated from the equivalent configuration inside my home MQTT environment.

What I’ve built here is basically an Alexa to MQTT bridge. This means I’m not limited to 1-to-1 control of individual lights. With the right MQTT messages in the device discovery JSON blob I could also control groups of lights, timed sequences, or anything else I care to code up.