Alexa Telemetry Data

When you build an Alexa skill you’ve got two rich sources of information describing how users are interacting with your skill.  The first is the data in the Alexa Skills Kit request object.  The request object is the JSON representation of what you said to your Alexa device. Alexa Voice Service(AVS) takes your spoken words and breaks them down into this object based on the interaction model you define.  This JSON is what’s sent from AVS to your custom back-end application.  The other rich source of data is whatever you decide to collect while your application is processing requests.  Let’s explore the request object and look at some key data points stored there.

Request

  • RequestID — a unique identifier for the request coming in. If you build a skill that has several steps, then each time the user speaks to their Alexa, a new RequestID is sent for each utterance.
  • Timestamp — self-explanatory.
  • Locale — a string describing your user’s preferred language. For example, “de-DE” would indicate the user has their Alexa set to German. “en-US” is United States american, and “en-IN” would indicate India English.  This data can be useful for customizing the experience to different language demands.  For example, If a user has selected “en-GB” and they ask for the closest multi-story”, they’re asking for a parking garage.
  • Intent Name — the name of the intent AVS understood.  Usually, you will have just a handful of intents. being able to measure how popular each of them are will help you decide where you want to invest more development time.
  • Intent Slot Names and values — remember these are the variables of your skill. Recording how often different values are used may help you decide to refine your utterances.

CatchAll Intent and Slot

What if you wanted to see what users were passing as Utterances that you never expected?  You can set up an intent called “CatchAll”  this intent has a single slot and utterance: “{CatchAll}”.  In your application, you fail gracefully when a user hits this intent.  You want to tell the user, “I didn’t understand you, but I’ve sent your request on for further processing.” Then, you can log all the utterances your users send to a log file.  Every so often, you can analyze this log file and look for patterns in the data.

You may find users are using a different phrasing than you expected. You might find that users are asking followup questions you never expected.  Use this data to improve your skill and increase the likelihood users will keep using your skill!

System

  • DeviceID — a unique identifier for the physical device accessing your skill through the Alexa service.
  • Supported Interfaces — is this a dot or echo and only have audio abilities, or is this a spot and has a video screen.  Knowing how many of your users have different abilities can influence your decision to create multi-modal interfaces for your skill.

Session

  • SessionID — for each conversation a unique ID is assigned.  This ID will accompany every utterance the user sends until the skill times out or the users cancels out.  This is useful for tracking your user “funnel”. Do you have users start ordering tickets through your skill, but stop before they complete the order?  Where are they stopping? Knowing this can help you refine your interaction model making it easier for your users to complete tasks.
  • Attributes — the Swiss Army Knife of the request.  These store key-value pairs during sessions.  When you need to “remember” data between steps, this is the closest destination for that data.  If the data needs to be persisted past a session, be sure you write out the data to something a little more durable, like SQL Server!
  • User Access Token –token used to identify the current user in another system, like facebook or twitter.
  • Permissions — if the user grants you permission to see their physical address or other permissions, this is where you can find the access token you’ll use to get to that sensitive data.

Internal Application Telemetry

Once the request makes it to your application, the sky’s the limit on what you can track.  Given a unique user ID, you could track how many times that user has used your skill. n You could track what time(s) of the day he or she uses the skill.  If you built a quiz skill you could track high scores by user, region, or any other metric you’re capturing.  You could also aggregate user scores to provide questions users would find more challenging.  Anything you want to track here, you can!

Analyzing this data

Collecting the data is pretty easy, but how do we analyze it?  you’ve got two choices.  You can transform the data into a format easy for you to analyze and store that in an OLAP database.  This is fine, so long as you can handle the insert volume.  You also run the risk of painting yourself into a corner by transforming the data at first.

The alternative is to store the data raw: in text or JSON format.  Leave the transformation until later, when you know what you’re looking for.  This option is a better fit for a data lake.  Data lakes can handle higher volume inserts since they’re supported by highly distributed file systems.  The cost here, is you have to transform the data into a format for analysis on demand.  This could cost more in CPU over the long term.

Either option is valid, just be sure to store the data so you can extract the intelligence from it later!  If you’re interested in more about how to analyze Alexa data from data lakes, check out my articles on Azure Data Lake.  I’ll be growing that content over the coming year as I collect more data from my Alexa skills.

In the meantime, if you have any questions, please let me know. I’m here to help!

Leave a Reply

Your email address will not be published. Required fields are marked *