Hacking Big Mouth Billy Bass – part 3/3

 

A few weeks ago there was a lot of interest on social media in a video showing a Big Mouth Billy Bass animatronic novelty device seemingly take on the role of Amazon Echo, including animating its mouth and body in sync with the speech. With my recent exploits in connecting strange crap to Octoblu I decided to have a go at automating and Internet-connecting Billy Bass.

In this three-part blog series I’ll cover:

  1. Reverse-engineering Billy Bass and automating its movements based on audio input
  2. Connecting Billy Bass to the Octoblu IoT automation platform and synthesising speech for it
  3. (This part) controlling Billy Bass with Amazon Echo and Slack

Recap and introduction

In the first part of this blog series I described how I added an Arduino to Big Mouth Billy Bass to enable it to be controlled by commands over USB-serial as well as automatically moving its mouth in sync with audio input. In the second part I described how I connected this to the Internet using the Octoblu IoT automation platform. In this final part I’ll describe how I used the Octoblu integration to control the fish from Amazon Echo and Slack.

Where we got to last time was an Octoblu flow that could take a message containing a text string, send it to the Voice RSS text-to-speech service, then send to resulting mp3 to the Billy Bass fish to play via its speaker and move motors in sync. The examples in this blog post look at how to generate a message with suitable text in order to feed this flow.

Amazon Echo

The goal here was to create a “skill” for Amazon Echo to allow voice control of the fish, e.g. saying “Alexa, tell Billy Bass to say hello world” would send a message of “hello world” to the text-to-speech service as described in the previous part of this blog post series.

In the Octoblu web interface I edited my “Billy Bass” flow to add a trigger node to act as a HTTPS POST endpoint for the Alexa scripts to call – I named this “Call from Alexa skill”. I connected this to a “Compose” node which is just there to tidy up the incoming message from the POST data into the form the rest of the flow is expecting: it creates a key named “text” with value “{{msg.data.text}}”.

billybassalexatrigger

I’ve previously created a smart home skill adapter for Echo – creating a custom skill is broadly similar so I won’t repeat all the details again but will highlight the key parts specific to fishy conversations.

Firstly I created a new Alexa skill in the Amazon Developer Console using the custom skill type and an invocation name of “billy bass”. I then defined an interaction model for the skill – this describes the forms of wording that Echo will recognise and defines slots that are, in effect, variables that will be populated when voice recognition takes places. The intent schema defines just one intent – asking the fish to say something:

The slot is the word(s) that we’re asking the fish to say. The slot can either be a constrained type (such as a date, or fixed, enumerated list) or, as is the case here, an open-ended list. I defined the LIST_OF_SAYINGS slot type as a custom slot type with three simple examples: “hello world | how are you today | what time is it”.

The final part of the interaction model is to provide some sample utterances which tell Alexa what kind of phrasing maps to the intents (in theory this doesn’t need to be an exhaustive list of all ways in which a user may state the intention). Here I used the single sample: “BillySay to say {Saying}” where “BillySay” corresponds to the intent and “{Saying}” is a placeholder for the open-ended slot.

Overall this looks like:

billybassechomodel

alexabpThe next step was to create an AWS Lambda function to handle the skill. This is called each time Alexa matches an intent on this skill. I used a Python 2.7 blueprint for an Alexa skill. The blueprint prompts to create a trigger from the Alexa Skills Kit to Lambda – let it create this. In the Lambda function configuration page I edited the code (see below) and set up the role – I used the same role as for my home skill adapter (see here for details).

The core part of the code is the on_intent function which simply takes the text that Alexa recognised as being the “Saying” slot in the intent in the schema defined above and sends it to my Octoblu trigger as a HTTPS POST with the text being in the “text” field of the posted message. The skill ID (“amzn1.ask.skill.<UUID>”) can be found in the skill details in the Amazon Developer Console. The Octoblu trigger URL can be found from the “thing inspector” in the Octoblu web interface having clicked on the relevant Trigger node (the “Call from Alexa skill” one here).

Having saved and published the Lambda function I added its ARN (this should be displayed on the Lambda function’s pages in the AWS console) to the Alexa skill via the configuration page in the Amazon Developer Console.

And that’s pretty much it. After restarting the Octoblu flow I was able to say to the Echo “Alexa, tell Billy Bass to say hello world” – this led to Alexa matching the skill and intent I defined above then calling the Lambda function with the intent and its slot value (the words the fish has been asked to say). This function then called Octoblu via a HTTPS POST to the trigger and this then got passed to Voice RSS for text-to-speech and then back to Octoblu to be routed to the connector running on the Raspberry Pi and ultimately to the fish itself. With the fish and the Echo next to each other you can get the fish to talk to Alexa, e.g. say “Alexa, tell Billy Bass to say Alexa what time is it” – the fish will say “Alexa what time is it” and then the Echo will answer.

Slack

I’ve been playing with Slack as a way to have human interaction with Octoblu flows. Adding a way to control the Big Mouth Billy Bass seemed like a natural next step 🙂

Luckily Octoblu already has a good way to connect to Slack using a streaming “slurry” thing. I set this up using a private test Slack account I already had by first creating a bot account and extracting an API token for it (see Slack docs for details on doing this). I added the bot to each Slack channel I wanted it to respond to. I then navigated Octoblu’s web interface to “All Things” and selecting the “Slack Streaming Bot (beta)” thing. Having clicked on this I was asked to authorize access and then to provide the bot’s token.

Having saved this thing I then went to my existing “Billy Bass” flow, added the new Slack thing and clicked the “UPDATE PERMISSIONS” button to enable Slack to send messages to this flow. I routed Slack messages through a non-null filter on “{{msg.data.text}}” to ensure there is a parsable Slack message in there, then through a function node to look for Slack messages that start with “Fish:” and extract the rest of the message to forward to to the text-to-speech service an ultimately to the fish itself. The second filter node removed any messages that didn’t match the “Fish:” prefix.

slackfish

So now I can type into a Slack channel “Fish: hello world” and the Billy Bass will say “hello world.

In closing

So there we have it – an IoT talking fish integrated with Amazon Echo and Slack using Octoblu, Raspberry Pi, Arduino, Voice RSS and a handful of electronic components. Clearly not the most practical IoT device ever created but hopefully an illustration of the power of these tools.

 

Advertisements

2 thoughts on “Hacking Big Mouth Billy Bass – part 3/3”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s