LORE Case Study

01 — The challenge

Most deaf children grow up with hearing parents — who start at zero.

Early sign language is critical for a child’s development — yet there are barely any digital, age-appropriate tools for preschoolers. For hearing families, even reading a bedtime story together becomes hard without a shared language. ELIA, an OST research project, asked us to explore what a first learning tool could look like.

~10’000

deaf people live in Switzerland

~90 %

of deaf children have hearing parents

0–4 yrs

critical window for language acquisition

Few

age-appropriate digital tools in DSGS

Sources: Swiss Deaf Federation (SGB-FSS), cited in swissinfo.ch, «Was gehörlose Kinder brauchen», 2018 · own scan of existing tools

Design question

How might we make the first steps into sign language playful for a young child — and doable for a family that starts at zero?

02 — Research & synthesis

We listened to families and experts first.

Desk research (30+ sources), a scan of 7 existing tools, a stakeholder map and 3 interviews with a parent, an expert in special-needs education and the Deaf community.

InspireEmpathizeDefineIdeatePrototypeTestIterate

Time-critical

The first months matter most — delays can’t be made up.

A family matter

Parents and siblings all need to sign; siblings are often left out.

Sign language is 3D

Static pictures make movements hard to read.

Joint attention first

Connect parent and child instead of isolating the child.

Visual calm

Busy backgrounds pull attention away from the hands.

Success motivates

Seeing that communication works keeps children going.

“There’s an app for everything. But for this — simply nothing.”
— Parent, research interview (translated)

03 — Personas & journey

Two users, one family.

Marina

Parent · learner

“I need a way for our whole family to learn — in everyday life.”

Flexible learning · include siblings · works offline

Finn

Young deaf child · beginner

“Learning signs shouldn’t feel like homework.”

Cool visuals · rewards · fast, visible progress

Opportunity

Families go through six stages from diagnosis to stability — mostly on their own. A digital companion could support them anywhere, anytime and from the very first day.

04 — Concept

Meet Lore: a story that only moves forward with your hands.

After a Crazy-8 round, the interactive storybook and the family duel scored highest. Together they became Lore — named after the Old English word for knowledge handed down.

Lore with 3D signing hands spelling LORE

Inspired by the wordless film “Flow”

Silence at the startOnly pictures carry the story.

Lore introduces each signA sign is used only after Lore shows it.

Vocabulary growsFrom A1, story by story.

05 — The solution

From story to sign — and back to the story.

iPad showing the Lore story map with clouds

Story playerPictures and short text; Lore appears with every new sign.

Sign stopThe front camera tracks the hands with a feedback ring.

Gentle feedbackGreen means correct — otherwise “Almost! Look again”.

Lore shows the sign for “cloud” – a child copies it in front of the iPad camera

Sign stop · camera tracking

Lore shows it — the child signs it back.

At each sign stop the story pauses. Lore demonstrates the sign, the front camera follows the child’s hands in real time, and only when the movement matches does the story continue.

iPad showing earned badges, Lore pointing at the screen

BadgesEvery completed story or letter earns a badge.

Finger-alphabet mapLetters unlock one by one, demonstrated by Lore.

Built for familiesParent info, offline mode and DSGS, LSF or LIS.

Story scene: Lore waving at the Explorer Bear

Everyday signs in context

Lore and the Explorer Bear move through daily routines — waking up, eating, drinking, going to the toilet, feeling tired.

Lore demonstrating four finger-alphabet signs

06 — Visual design & design system

A calm sky at sunset.

Busy backgrounds distract from the hands, so the interface stays soft and quiet. Tokens with light and dark mode, Kodchasan for headlines and Inter for reading text.

Marine Blue#193CF5

Light Blue#0D6CF0

Lilac#7B8AFF

Pink#F5B0FF

Rose#FDCDE8

Ink#41383D

White#FFF8FC

Headlines · Kodchasan

Rounded & friendly

Reading text · Inter

Neutral & legible

07 — Character & physical prototype

A companion and a stage.

Lore as a boy and as a girl, both with visible hearing aids

Lore — boy or girl

Modelled and animated in 3D, with adjustable skin tones and visible hearing aids — so every child can see themselves in the story.

Wooden story theatre holding the iPad like a stage

Story theatre — cardboard to wood

A cardboard version tested proportions and camera height. The final wooden stage is sturdy, non-slip and keeps the camera at a child-friendly angle.

Process · behind the scenes

From storyboard to video – and from cardboard to theatre.

A look at the raw process: the storyboard for the video I generated with AI (Google Flow), and how the story theatre evolved from a cardboard prototype into the final build.

Storyboard for the LORE video — **Video storyboard.** Scene by scene before generating the clips with Google Flow.

Story theatre from cardboard prototype to final version — **Story theatre.** Cardboard prototype → final version. Testing showed it needed to be sturdier and fix the iPad angle.

08 — Usability test & iteration

Tested with a child from the target group.

A moderated test with a child from the target group in a familiar setting — one story, three sign stops.

What worked

Started and finished the story independently
Camera principle understood quickly once explained
The reward animation made the child laugh
Added signs on their own, without a prompt

What didn’t

Unclear that the camera meant “your turn”
Recognition depended on posture and angle
Story pace too slow for an energetic child
A 3D signer wasn’t instantly read as “copy me”

“Your turn!” moment

Lore points to the camera; a preview shows how to sit.

Stand fixes the angle

The theatre holds the iPad at the right height.

Faster story flow

Livelier pacing and more sign stops in the coded version.

Real signers

Future stories add people alongside Lore.

09 — My contribution & learnings

What I built — and what I take with me.

Technical Lead & Media Designer

Started as Creative Officer and shaped the early visual direction
Built the camera-based sign recognition from scratch; a teammate refined it for the final version
Created the project videos with AI (Google Flow)
Co-developed the logo and helped keep Lore’s look consistent across all media
Research, ideation and testing together with a team of four

Test early with the real audienceOne session taught us more than weeks of assumptions.

Technology must adapt to the childRecognition and timing have to fit a lively four-year-old.

Physical and digital belong togetherThe theatre solved a UX problem the screen alone couldn’t.

Next stepsGames sectionReal signers in storiesDifficulty levelsDeeper gamificationFull dark modeLSF & LIS

Thank you for reading.

Lore is a tribute to non-verbal communication — because everyone deserves to be seen, understood and included.

2nd placeDigital Design Festival 2026 NominatedYoung Innovators Award 2026

ELIA project at OST ↗Clickable prototype ↗Project video ↗Coded story ↗Alphabet demo ↗Design system ↗

Michelle Schubiger · UX/UI Design Next project: Progrès →