Hacking Big Mouth Billy Bass – part 2/3

A couple of weeks ago there was a lot of interest on social media in a video showing a Big Mouth Billy Bass animatronic novelty device seemingly take on the role of Amazon Echo, including animating its mouth and body in sync with the speech. With my recent exploits in connecting strange crap to Octoblu I decided to have a go at automating and Internet-connecting Billy Bass.

In this three-part blog series I’ll cover:

  1. Reverse-engineering Billy Bass and automating its movements based on audio input
  2. (This part) – connecting Billy Bass to the Octoblu IoT automation platform and synthesising speech for it
  3. Controlling Billy Bass with Amazon Echo and Slack

Recap and introduction

In the first part of this blog series I described how I added an Arduino to Big Mouth Billy Bass to enable it to be controlled by commands over USB-serial as well as automatically moving its mouth in sync with audio input. In this instalment I’ll describe how I connected this to the Internet using the Octoblu IoT automation platform.

The Octoblu connector

“Things” can be connected to Octoblu via “connectors”. Connectors run somewhere that they can talk to the “thing” as well as talk to Meshblu, Octoblu’s messaging platform. In this case my connector runs on a Raspberry Pi. The overall operation of the connector is that it listens for incoming Meshblu messages which contain a base64 encoded mp3 file. Each received message is added to a local FIFO queue (to avoid trying to play multiple messages simultaneously). When a message is played from the queue the connector first sends a USB-serial message to the Billy Bass Arduino to tell it to move the fish’s head. It then creates a temporary mp3 file and base64-decodes the mp3 content from the message into it. Next the connector spawns an external mp3 player program on the Pi which plays the mp3 file and on completion the callback in the connector gives the fish’s tail a quick flip before returning all body parts to their at-rest position.

I created the connector using a simple node.js program based on the v1 (now deprecated) Meshblu NPM library (“npm install meshblue@1.34.1”). This is very much prototype code with hard-coded items and without much error handling. To use the meshblu library I needed to create a “generic device” from Octoblu’s “All Things” list: having clicked on “GENERIC DEVICE” in the “OTHER” section, I clicked the “CONNECT GENERIC DEVICE” button; leaving the selection as “Register a new thing”, entering a name for the device (“My Billy Bass” here) and leaving the “type” field empty; I clicked “CONNECT GENERIC DEVICE” again; and found the new “My Billy Bass” device in the list of my things. I clicked on the device to bring up its configuration page. From here I copied the UUID, and clicked “GENERATE TOKEN” to generate and show an authentication token – I copied this also. I then added both the UUID and token to a file named meshblu.json (see the comment in the code for an example) alongside the connector.

connectorsetup

As well as meshblu, I also needed to npm-install child_process, fs, serialport and tempfile. Having connected the Arduino to the Pi and ensured its serial interface was available as /dev/ttyACM0, and having connected the fish’s audio input (this goes to the amplifier and to the Arduino’s analogue input) to the Pi’s headphone socket, I ran “node piaudio.js”. This confirmed I was connected to Meshblu but this won’t do anything useful without an Octoblu flow.

piaudiojs

The Octoblu flow

So now I’ve got a fish that can talk and move its mouth, head and tail by playing an mp3 file sent to it via Meshblu. To make this do something worthwhile meant building an Octoblu flow to create a suitable mp3 file containing synthesised speech. To do this I used the text-to-speech web service Voice RSS. This service provides an easy-to-use API service to turn text into speech in a variety of formats, languages and accents. It has a free tier that allows for up to 350 requests per day. I had looked at doing text-to-speech locally on the Pi using a tool such as Festival, however I found the Voice RSS output to be much higher quality, and using a web service allows for more future flexibility than having everything locked down on the Pi.

The flow begins by generating the text that Voice RSS will later convert to speech. In this flow I have two ways to do this: firstly, for testing, I am hard-coding two test phrases using “compose” tools (these are just assigning the text strings to a key named “text” that will later be extracted in the HTTP API call to Voice RSS). The second mechanism is to receive the text via a HTTPS POST call into an Octoblu trigger tool (it’s named “Call from Alexa skill” in this flow – you’ll see this in use in the video at the end of this post as well as in the third part of this series). This then gets put into the same “text” key as above.

billbassflow

The Voice RSS API is very easy to drive from Octoblu. I simply used a HTTP GET tool configured with the various API parameters, including my voicerss.org API key. The text to convert is taken from the “text” field of the incoming message using Octoblu’s “moustache” syntax (“{{msg.text}}”).

voicerss

collatemp3The output from the HTTP GET is base64 encoded mp3 which is carried in the message as the only payload. The “compose” tool that follows the HTTP GET simply puts this payload into a key named “mp3” to match what the connector is expecting (the reason for doing this is to allow extension with other data/metadata, such as customised motor commands, in the future). This is then sent to the “My Billy Bass” device (the one I configured earlier) with the “Use Incoming Message” option enabled (this causes the message we just created with the mp3 data to be sent directly to the connector).

Summary

Having connected my Arduino-connected Big Mouth Billy Bass to a Raspberry Pi and to Octoblu I now have a HTTPS POST webhook I can call with some text that will be sent to Voice RSS to be synthesised into speech. This will then be sent to Billy Bass via my Meshblu connector and the fish will speak and move its mouth, head and tail to sync with the speech.

In the third and final part of this blog series I’ll use these mechanisms to connect Billy Bass to Amazon Echo (as in the video below) and Slack.

Code

 

 

 

Advertisements

3 thoughts on “Hacking Big Mouth Billy Bass – part 2/3”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s