Using LightwaveRF switches to control other stuff

img_9028I’ve been using LightwaveRF‘s range of wirelessly controlled lighting and power outlets for some time and I’ve built up a collection of switches and remote controls for them. These switches broadcast control messages, formed of a command (e.g. on/off) and a unique identifier for the transmitter, using a simple on-off keying modulation on 433.92MHz with each light fitting or other device being “paired” to one or more transmitters. In reality the pairing is actually just telling the device which command(s) to respond too – the transmitter units can’t receive anything which prevents a bidirectional pairing. This all runs in parallel with the LightwaveRF network bridge which is really just an IP-controlled 433.92Mhz transmitter.

With my home automation system now growing to include Philips Hue and various homemade devices I’d like the LightwaveRF switches and remote controls to be able to control more than just LightwaveRF devices which means taking them our of their constrainted world of 433.92MHz. To do this I am building a set of 433.92MHz receivers to forward the command messages to my MQTT broker. This means I can extend my existing set of automation rules to react to the LWRF switch commands by, for example, sending a command to the Hue Hub to control a Hue light. I can also build up more complex operations with other rules such as only performing certain actions outside of daylight hours, or controlling multiple devices at the same. This also allows for more interesting control of the LightwaveRF devices themselves because I can now insert rules between the transmitter and receiver simply by not pairing the device with the transmitter and instead routing everything through the MQTT world and the LightwaveRF network bridge.

img_9021The prototype is based on three main components:

  1. A 433.92MHz receiver – I’m using a cheap one from Amazon. This has a 17.3cm wire antenna soldered on to it.
  2. An Arduino Uno R3 to handle the demodulation of the on-off keying and the protocol decoding.
  3. An existing Raspberry Pi to forward the decoded messages from the Arduino to the MQTT broker.

Why the separate Arduino? That’s because I don’t then have to worry about responsiveness of the decoding software if it was to be running on a Raspberry Pi alongside other stuff – some of which may also be latency sensitive.

There is plenty of material online (see the resources below) about the LightwaveRF protocol. I found that none of the existing code (at least what I could find) worked in the way I wanted so I implemented a decoder from scratch using the various online protocol resources as guides.The decoder, running on the Arduino, is formed of three main pieces. Firstly a demodulator that looks for state changes on the data coming from the 433.92MHz receiver. This then feeds the demodulated bits into the main protocol decoder which looks for the message start bit, each byte’s start bit, and a valid pattern for the byte (which is a form of 4b8b encoding). The final piece kicks in once the transmission has ceased for 25ms: the buffered received data is de-duplicated and for each unique message a MQTT message is sent over the serial port (via USB).

The format of the MQTT message is topic=”LWRF433/<unitID>/<channel>”, message=”<command 0=on,1=off>|<level>”, where:

  • <unitID> is the unique 3 byte identifier for the switch unit
  • <channel> is the channel number, the meaning of which varies across the different type of switches
  • <command> and <level> tell the light what to do and vary depending on whether the controlling switch is a simple switch, dimmer, or mood switch.

lwrfmqtt

The initial prototype is connected to an existing Raspberry Pi running LibreELEC/Kodi as a media centre. The Arduino is connected to a USB port on this Pi. The function of the Pi is simply to forward MQTT messages received over the USB serial port from the Arduino to the house MQTT broker. To do this in the LibreELEC/Kodi environment is a little trickier than in a general Linux/Raspbian environment due to the locked-down, read-only nature of the LibreELEC distribution. However the Kodi add-on mechanism provides a reliable way to do this.

20161009_170439I created a Kodi add-on formed of two main parts. Firstly a Python script that acts as the main forwarding loop between the USB serial port and the MQTT broker. This is based on code from a previous, albeit standalone, proxy doing largely the same thing – it includes a device scanning function to watch for Arduino devices appearing and to fire up proxy threads for each one – this was originally done to avoid static configuration and to allow for hot-plugging of new Arduino devices (this was in a far more complex, multi-Arduino system, compared to this simple case).

The second part of the Kodi add-on is really just boilerplate and plumbing to kick off this script.

So putting it all together this is how it works:

  1. A button is pushed on a LightwaveRF switch, this sends a message (actually a burst of several) on 433.92MHz.
  2. The 433.92MHz receiver module receives these messages and passes the raw data to the Arduino.
  3. The software on the Arduino decodes the messages, de-duplicates and sends one or more MQTT messages over the USB serial port.
  4. The proxy script running on the Kodi Pi forwards the MQTT message(s) to the MQTT broker.
  5. Part of my MQTT rules engine is subscribed to LWRF433 messages and receives the message(s). After debouncing, the relevant rule is invoked, e.g. sending a separate MQTT message to control a Hue light.

 

The initial use-case is to use the LightwaveRF remote control to turn off the room lights, pause whatever the media centre is playing, and turn off the TV, all with a single button push. To do this I created a rule that on receipt of the appropriate LWRF command via MQTT sends out two more MQTT messages, one to turn off the TV and one to pause Kodi. Both of these are handled by a separate add-on running on the same Kodi Pi, but that piece of functionality is completely separate to the LWRF add-on described above and is the subject of a future blog. The room lights turn off directly in response to the LightwaveRF 433MHz command without any MQTT infrastructure being used.

lwrfrule

Resources:

Advertisements

Controlling custom lighting with Amazon Echo and a skill adapter

ctb2adfxeaaptil-jpg-largeAmazon launched Echo in the UK today – it’s been a long wait! My pre-ordered Echo arrived this morning and my first priority was to get it (her?) to control my home lighting. As I’ve previously written about, my LightwaveRF lights are currently managed by a custom set of scripts communicating within the house using MQTT pub-sub. This means that Amazon Echo (or Alexa, as I’ll refer to it/her from now on) doesn’t know how to interface with them like she does with Philips Hue, for example.

Luckily Amazon has made available its Smart Home Skill API which allows individuals and home automation vendors to provide “skill adapters” for the various home automation “skills” Alexa already has. This means it is possible to say “Alexa, turn on the bedroom light” and have Alexa use whatever underlying system and technology you have to execute the command. This is preferable to defining a new custom skill because it avoids the need to use a skill trigger word (e.g. “Alexa, turn on the bedroom light using LightwaveRF”). AWS Lambda is used to provide the concrete implementation for the abstract actions.

octoblualexaIn my case the skill adapter will make a HTTPS call to an Octoblu trigger node passing details of the required action (essentially just a MQTT message with the topic being the light to control and the message body being the action (on or off)). The Octoblu flow then messes about a bit with the JSON structure before passing the message to an existing Meshblu device that connects my home MQTT world with Octoblu. In reality I’m using Octoblu and Meshblu here as firewall-bridging plumbing to get a MQTT message from the Lambda function into my home environment.

Having signed up for my Amazon developer account I followed Amazon’s steps to create a Smart Home Skill. This started by creating a new skill (of type “Smart Home Skill API”) in the Alexa Skills Kit section of the Amazon Developer Console – I chose to name this skill “James’s Lights”.

To provide the actual implementation I created a Python 2.7 Lambda function named “JamesLightsSkillAdapter” in AWS (using the eu-west-1 (Ireland) region to co-locate with the Alexa voice service for the UK) based on the alexa-smart-home-skill-adapter blueprint. I based the code on the template provided in the “steps to create” page above. For the role I selected “Create new role from template(s)”.

The code handles two separate types of message from the Alexa service:

  1. Device discovery – this is an action initiated by the end user from the Alexa app (or by voice) to get an inventory of devices that Alexa can control. In the Lambda function this is implemented by returning a big blob of JSON with an entry for each device. The “friendlyName” item being the words Alexa will recognise to control the device. I’m using the additionalApplianceDetails item to record the MQTT topic that will be used to control this device. My initial prototype implementation hard-codes the entire inventory in the Lambda function – clearly not a long term solution!
  2. TurnOnRequest and TurnOffRequest commands – these are issued by the Alexa service when Alexa is asked to turn a device on or off and the device is recognised as one in the inventory. The Lambda function is called with the relevant JSON blob and therefore my code can pull out the previously configured MQTT topic and send that as part of a HTTPS POST to the Octoblu trigger mentioned above.

I tested the Lambda function using the “Alexa Smart Home – Control” sample event template, manually crafting the JSON messages to match the ones the Alexa service will be sending. After testing it’s important to make sure to enable the Alexa trigger for the Lambda function.

echoskill

Back in the Developer Portal I configured the skill to use an endpoint in AWS Europe using the Lambda ARN created above. As this will be a private skill adapter I didn’t really need account linking (this is how a regular user would link their Echo with, for example, their Philips Hue account) but the console wouldn’t let me proceed without setting it up. Therefore I followed this great blog post’s instructions on how to use Amazon itself as the linked account.

file-28-09-2016-21-56-25Having saved all the skill configuration and enabling testing I then used the Alexa app on my iPad to add this newly created skill (in the “Your skills” section) and logged in with my Amazon account as the account linking step. From there is was a simple matter of letting the app and Alexa discover the devices and then Alexa was good to go.

I can now say to Alexa: “Alexa, turn the lounge light on” and a second or so later it’ll come on and Alexa will say “OK”. What happens under the hood is more interesting though:

  1. The Alexa cloud service processes the message, figuring out its a smart home control command
  2. The Alexa service looks through my discovered device list and identifies the “lounge light” is one that is controlled via this skill adapter.
  3. The Alexa service makes a call the my AWS Lambda function with a JSON message including the configuration for the requested light as well as the “TurnOnRequest” command.
  4. My Lambda function makes a HTTPS POST call to the Octoblu trigger with a MQTT-like message including the topic string for the requested light and the “on” message.
  5. The Ocotblu flow forwards this message via Meshblu to a simple Meshblu connector I have running at home.
  6. My Meshblue connector publishes the messaage to my local MQTT broker.
  7. The LightwaveRF script also running at home and subscribed to “Light/#” messages picks up the message and looks up the room/device codes which it then send via UDP to the LightwaveRF bridge.
  8. The LightwaveRF bridge sends the appropriate 433MHz transmission which is picked up by the paired light fitting and the power is switch on.

As this is all highly customized to me I’ll be leaving the app in its testing state, not putting it forward for certification (which would certainly fail!) and publication.

Future work

Right now the implementation hard-codes my device list as well as the URL of my Octoblu trigger. I’d like to, at the very least, make the device list dynamically generated from the equivalent configuration inside my home MQTT environment.

What I’ve built here is basically an Alexa to MQTT bridge. This means I’m not limited to 1-to-1 control of individual lights. With the right MQTT messages in the device discovery JSON blob I could also control groups of lights, timed sequences, or anything else I care to code up.

Getting started with home automation

20160908_195003I’ve always had an interest in electronics and software, and in connecting the two together. I guess it started in my pre-teen years when I figured out I could use the cassette motor relay on my family’s BBC Micro to turn LEDs and buzzers on and off, and it grew as I learned to use the BBC’s 8 bit “user port” with more sophisticated electronics. I’ve used a variety of technologies and gadgets over the years, including various microcontrollers (PIC in particular) accessed via serial (RS232 or RS423), Elexol’s USBIO24 GPIO devices and, more recently, various Arduino and Raspberry Pi models. I’ve built a number of different things such as controls and displays for my flight simulator, computer control and feedback interfaces for my model railway, and miscellaneous gadgets of varying levels of utility. But until recently an area I’ve not explored is home automation – a topic well aligned with my experience and interests. That changed about 4 years ago when the problem of needing landlord permission to run cables and drill holes disappeared and I became a homeowner.

Like most people who venture into home automation my starting point (and mostly still the extent of my experience) was lighting. The automation of lights is both relatively straightforward and safe – there are no moving items under control of my software so no bug can lead to overheated motors or physical damage.

20160908_185431A decade ago I’d probably have started by buying a bunch of relays or triacs and working back from that (in fact I still have a device much like this that I fear to plug in) however I decided that it was time to leave the mains voltage to the professionals and concentrate on the low voltage stuff and the software. When I looked around for suitable hardware I was particularly impressed by the LightwaveRF range. The core of the range is a set of dimmers and relays in various form-factors including inline (which for the light switch types mean they need a leakage current through the lights to power them – something I’m not keen on) and plug-in units. All the dimmers and relays are controlled by 433MHz RF, either locally from handheld remotes or wall-mounted buttons, or via the network using an Ethernet to RF bridge (called a “Wi-Fi Link” despite having no Wi-Fi capability!).

20160908_185332The “Wi-Fi link” is designed to be used with LightwaveRF’s cloud service to enable control of the lights from mobile apps or via REST APIs. However, it can also be used directly by sending suitably formatted UDP packets to it.

When I first started using LightwaveRF my priority was to use the system to replace timer-based security lights simulating occupation while away from home. I initially started with Paul Clarke’s excellent “lightwaverf” tool which can drive the Wi-Fi Link from either command line invocations or from events in a Google calendar. I ran this tool on an early-model Raspberry Pi sitting in the cellar. I coupled this with an IFTTT recipe that runs daily and populates a dedicated Google calendar with a sunset event which was the basis for triggering the switching on of some of house lights.

20160908_185404This was all fine until Google withdrew the ability to share a calendar privately as an XML download. This was the motivation to build a more flexible system that would enable more use-cases for automatic light control (these will be the subject of future blogs). I looked at using Octoblu, an IoT automation platform acquired by Citrix. It didn’t have support for LightwaveRF but I did end up building a simple gateway to allow me to use the two technologies together. More importantly it was, at the time, not particularly stable (it’s much better now) so I ended up building a local MQTT-based system with a bunch of individual services connected via the MQTT broker (all running on the same Raspberry Pi as the previous system). One of these services is the LightwaveRF service which subscribes to messages about light controls and sends the appropriate UDP commands to the LightwaveRF Wi-Fi Link device (turns out there’s a NPM library for this). Another service is the calendar service which uses the Google calendar APIs to regularly pull down the same automation events as I was using with the previous system, but this time using my Google account and an API key, thus avoiding the problem of the missing private XML calendar. This service then issues lighting control command via messages published to the MQTT broker. By adopting the same calendar format as used previously I was able to continue to use the IFTTT recipe to populate the sunset events and enable/disable the security lights based on my travel plans

I’ve since added more services to the MQTT environment enabling more lighting automation use-cases – keep an eye out for more blog posts on these in the future.