Fizbit

A chatbot for kids with autism

Task

Design a chatbot to engage with children with autism and model good conversational behaviors

Background

Sidekicks is an assistive technology startup focused on helping parents and caregivers of children with autism communicate with their kids through special interests, also known as affinities.

 

While the focus was on parent and child communication, if a parent wasn't available, we wanted something that could engage the child while we contacted the parent to chat.

Summary

I designed the interactions for a chatbot for children with autism, from bare-bones rules-based modules to interlocking conversations prompted by keywords. We went through multiple iterations, relying on research, in-person feedback, and analysis of user behaviors to shape our strategy.

Process

In our first version of Fizbit, we had three major goals:

Fun

Emotional

Identification

Engagement

As we worked on how to address these goals, we looked into other autism robots on the market for inspiration. Many of them had a series of modules that could be played in different orders, some fun like a "dance" and some instructive like identifying an emotion based on a face.

For Fizbit, we designed five modules, skewed more towards fun and engagement than learning. We wanted to prioritize bonding with Fizbit instead of education at the beginning.

Jokes
Fun Videos
Trivia
Emotion Lessons
Info
Fizbit Modules

The most complex module we designed was the Emotion Lessons. In creating the lessons, our priority was that the child would succeed every time. While we didn't expect they would always get the right answer, we put a focus on hints and explanation rather than "right" and "wrong." Below is the flow for if a child picks incorrectly.

Emotion Lesson Flow

I’m a paragraph. Double click me or click Edit Text, it's easy.

I’m a paragraph. Double click me or click Edit Text, it's easy.

DESIGN

How do you think Simba is feeling?

Storytelling

I’m a paragraph. Double click me or click Edit Text, it's easy.

My emotion sensors say we should look at Simba's face!

Consulting

I’m a paragraph. Double click me or click Edit Text, it's easy.

When Simba is singing about how excited he is, he has a big smile on his face.

Smiling is one way to show that you're happy!

Play a video
Ask what emotion
Play again + pause with a hint
Offer an explanation

We were able to get feedback both directly from our coworkers whose autistic children played with Fizbit and indirectly from the data of chatbot users. 

From the direct feedback, we learned that children enjoyed the sounds Fizbit made and liked watching the videos, but either had less interest in the emotion lessons, jokes, and trivia, or figured out how to put in the correct answer or skip the module to get back to what they enjoyed more. One of the kids didn't seem to understand the jokes, but would laugh because Fizbit laughed. 

From the indirect feedback, we found that most children interacted with Fizbit while waiting for another Sidekick (one controlled by a person) rather than choosing to interact with him. This was frustrating and confusing to some users, who would quit the app or return to the main menu multiple times to try to pick a different Sidekick.

The app itself was garnering positive reviews from parents, so for our next iteration we did a deep dive into the behaviors of power users to see how to complement the experience of a parent-controlled Sidekick with Fizbit.

Sidekicks has made him more confident & more social with his peers ... It is a great addition to all his other therapies and activities.

Did you know that after our sessions, my son asks to do Sidekicks every evening?

I can't imagine life without it! ... This gives my son -- and us -- one more way to be together.

We looked at two power users in depth to get a better understanding of how kids and their parents interacted with our Sidekicks. 

We worked with Dr. Erik Gregory on a grounded theory exercise where we listened to sessions with power users, therapists, and their parents and made observations about what we heard.

We combined our notes to trace trends and sort our observations into four categories:

Support

  • Redirection

  • Family presence

  • Rephrasing

their world

  • Relationship with technology

  • Friendship

  • Sharing interests

  • Focus on the everyday

  • Personal projects

Personality

  • Humor

  • Special interests

  • Voice / vocal cues

  • Emotions

  • Movement / activity

Engagement

  • Focus / distraction

  • Excitement

  • Interests

  • Personal projects

Listening to these live sessions, it was apparent that a preset schedule of activities didn’t have enough flexibility to follow the child conversationally. In-person family support was also much more present than we anticipated. Adding functionality for more responsive conversation could allow the child and parent to interact with Fizbit in a more engaging way.

As we moved forward in our design, we focused on following the interests of the child. Whether it was sharing about their day at school, telling a joke, or explaining the activity they were doing (one kid played piano for his Sidekick), we decided the most important goal of the chatbot was to listen and respond with interest.

Fizbit

V2

For the second version of Fizbit, we shifted our goals away from lessons and towards organic skill-building:

Conversation Skills

Connection

Designing Interactions

When we first started writing interactive conversations for our chatbot, we thought of them as full conversations - 6 to 10 turn exchanges with limited branching and a lot of input from the chatbot. Below is an example of an early conversation:

In internal testing, we quickly realized that this form was too static to be truly conversational. By trying to anticipate a few paths for branching, we were making large assumptions about how children would respond.

 

In our research on the vocabulary of autistic children and adults, we found that they were more likely to have an idiosyncratic vocabulary compared to neurotypical peers and compared to each other. This meant that assuming how any autistic child - let alone generalizing expected word choice across a large group of autistic children - didn't make sense.

Just as we prioritized process over end goals in our Emotion Lesson design, we wanted autistic children to feel successful in every conversation they had with Fizbit, whether or not they followed its lead or forged their own path while chatting.

To achieve this, we changed the format of our conversations to be like interlocking puzzle pieces. We had more conversations per topic that were shorter and could be played in any order. Below are some examples of how conversations could be experienced:

How old is your cat?

Do you have a cat?

Who feeds your cat?

What's your cat's name?

Interlocking Conversation Examples

Fun

Next Steps

“Start Listening” or “Pause" button

Many parents would repeat, explain, or discuss questions Fizbit asked with their children, which was very exciting! Unfortunately, this meant that the recording would cut off before the child could give their answer.

​Longer wait times on the recording would either make the recording way too long or cut off the beginning of when someone started speaking. A button that would either "pause" the recording or “start listening” only after it was pressed would allow for more time for fruitful and fun discussion.

Time and Day dependent conversations

Something commonly requested by parents was the ability to help with daily schedules, and Fizbit could become an excellent tool for this. By including short reminder conversations, Fizbit could keep kids on task as well as answer questions if the child forgets what is next, or is soothed by being able to ask about scheduling many times.

It would also allow Fizbit to ask about weekday vs. weekend activities, differentiate between "good morning" and "good evening", and allow it to have themed conversations for holidays.

Parent input and feedback

Parent input and feedbackCurrently, Fizbit's conversations and knowledge base are defined only internally. To better answer questions and personalize content, parents should be asked for feedback in a low-touch way. 


One idea is emailing parents a list of three questions asked by Fizbit and three answers that it didn’t recognize, and asked to provide text that Fizbit should say. These would be read and incorporated by the content team to speed along the process of personalization.

Mood check in

For older kids who may need some help keeping track of their feelings throughout the day, Fizbit could check in at regular intervals to see how they’re doing. This could track patterns in mood throughout the day and the week, encourage self-reflection, and help kids get help if they are stressed or anxious.

Please reload

Do you have a cat?

Do you have a cat?

What's your cat's name?

Do you think cats have nine lives?

Do you have a cat?

What's your favorite kind of cat?

Do you think cats have nine lives?

What's your favorite kind of cat?

What's your cat's name?

Do you think cats have nine lives?

What's your favorite kind of cat?

What's your cat's name?