💡Insight 1: Language is the "eye" for them to make sense of the world.

In daily life, visual information still dominates. Visually impaired people would use visual words, such as color words, for communication purposes. They learn the meaning of visual words through the context of language input and use this to understand the world. Language has become their "eyes" to see the world.

💡Insight 2: Making sense of words is harder.

Based on Piaget's theory on cognitive development, children aged between 2 and 7 years old develop symbolic thinking, which is marked by deferred imitation, symbolic play, drawing or graphic images, mental images, and more mature language in increasingly complex order.

First Language Acquisition Stage

6 months

Pre-linguistic stage / Cooing

Cry, make cooing sounds and utter nasal murmurs as their vocal tracts develop

8 months

Babbling stage

Babble, making noises and syllables that aren’t yet words

1 year 6 months

Holophrastic stage

Say single words (dada) that describe objects or identify their basic needs

2 years

Two-word stage

more food, doggy small, thank mom

2 years 6 months

Telegraphic stage

The cat stand up on the table

5 years

Rare and Complex Constructions

10 years

Mature Speech

In the study of Kephart et al. (1974), the spontaneous responses of blind and sighted people to a body concept game showed that blind children mentioned fewer body parts at every age when constructing an imaginary friend. Although tactile and auditory modalities were important to the blind, fingers and ears were not mentioned.

For blind children, the restrictive environmental information would hinder their concept development, making full sense of words and expressions that are critical to them.

💡Insight 3: Lack of knowledge of Chinese character morphology hinders language learning.

Chinese Braille is similar to a translation of the Chinese phonetic system, which means it does not carry the rich information in the graphic structure of Chinese characters

For sighted children, it is much easier to understand meaning of words with the combination of phonetics, morphology and semantics, but for BLV children, they lack an important dimension of morphology. Therefore, it is much more difficult to distinguish between homophones and learn the cultural connotation behind Chinese.

For example, Yong'E (ode to the goose) is the first poem that most Chinese kids learn, as it's pretty simple in context. However, BLV kids would have much difficulty to figure out what the title even means, as the braille would be the same for "brave goose" or "swimming goose."

💡Insight 4: Current teaching tools are limited, basically hand-made by teachers.

There is a lack of intelligent educational products for visually impaired children. At Shanghai Blind School, the teaching tools are generally handmade by teachers, which is inconvenient and limited.

Braille Learning Tool

Tangible Curriculum

Persona

Competitive Analysis

Current designs primarily focus on tactile interaction, using building blocks, imitation, cards, and touch pads as the medium. Auditory interaction, the dominate way of language input for BLV people, is neglected.

Design Requirements

Voice Game with Tactile Board

TaleVision, our design concept, is a combination of voice game and tactile interface. The child hears the story with interaction guidance from a cute virtual agent, and touches the board to feel the pixel image and braille of the relevant key words of the story simultaneously. The supervisor, teacher or parent, can input multiple stories (like textbook lessons or fairy tales), check the key words automatically selected, and oversee the pixel image generated by AI.

To achieve high resolution of tactile pixels, we referred to magnetic-based tangible user interface based on MagneShape () and SuperMagneShape (), where the repulsive magnetic force lifts the pin, forming a touchable image.

Initial Design

Iterated Design

To accommodate image representation for more recognizable pixel image, we increased the resolution from 25×15 to 40×24 and enlarged the board accordingly.

3D Printing & Hardware Control

Virtual Friend

Story Experience

Hi! Welcome to the BlossomLand. I'm Joy, the Guardian of BlossomLand.

(no action or press ↩️)

I’ll be with you every step of the way. If you need any help, just press the microphone button and call out “Joy,” and I’ll be there to assist you. The microphone button is on your right, and it has a picture of a microphone with a long round top and a handle beneath it. Can you press the button and tell me your name? (sound cue)

(press🎙️) I'm Clara.

Hi Clara! I love your name, it’s so cute! Now, let’s start our adventure! Are you ready?

(press🎙️) I’m ready!!

Follow me! On your right, there are four small buttons. The arrows on them show the direction to move. You can press the buttons just like I do and walk with me! (sound of the little fox walking)

⬆️(press ⬅️➡️ following sound direction)

Here we are! This is the grand gate of Peach Blossom Land. Can you feel it?

(Feel the tactile panel with the gate)

(After a pause) Now, can you help me open the gate? To open it, press the Enter key, which is in the middle of your right side. It has a curved arrow on it. Can you find it?

(Press the wrong key)

Oops! Seems it’s not the right one. Would you like to try again, or ask your mom or dad for help?

↩️

Yay! You did it! The gate opens! (Sound of the gate opening)

Yay! You did it! You’re such a great adventurer! Let’s go inside…

⬆️ or ↩️

Long, long ago, there was a little cat named Naughty.

(Feel the tactile panel with a cat)

One day, Naughty went out to fish. He came to the riverbank and rowed his little boat.

(Feel the tactile panel with a boat)

Naughty rowed and rowed, and finally, when he reached the end of the river, he saw a mountain.

(Feel the tactile panel with a mountain)

Yay! You “Hmm, this mountain is so beautiful! I’ve never seen anything like it before,” Naughty said amazedly. He was thinking—should he climb up to take a look, or keep rowing?

I think keep rowing is better. Otherwise he could get lost.

That seemed like a wise choice! So Naughty kept rowing and rowing.

(Feel the tactile panel with a boat)

As he did, he saw the mountain again. “What’s going on? This is so strange!” Taotao scratched his head, puzzled.

(Feel the tactile panel with a mountain)

“What’s going on? This is so strange!” Naughty scratched his head, puzzled. He decided to take a look at the mountain...