VR language learning
A virtual reality journey through Vancouver: an English conversation trainer where learners talk to digital avatars, powered by speech-to-text, an LLM and text-to-speech.
3 playable scenarios · positive survey feedback
Screenshots from the thesis flyer, ZHAW School of Engineering.
01
The idea
- English is the common language when people don’t share a native one.
- It runs most international companies and most of the internet.
- VR makes practising it immersive instead of a textbook exercise.
02
Teaching method
- Compared teaching methods for language learning in VR.
- Best fit: communicative tasks.
- The learner talks to digital avatars to get something done.
03
How it works
- The learner speaks to an avatar.
- Speech-to-text transcribes what they said.
- An LLM writes the avatar’s reply.
- Text-to-speech speaks it back.
- Conversations are free-form, not scripted.
04
The prototype
- Three playable scenarios, picked from a main menu.
- Adjustable difficulty.
- Restaurant: order food.
- Hotel: ask for directions.
05
Outcome
- A survey collected feedback on the prototype and the method.
- Positive overall.
- Shows potential for language teaching.
- Pointed out the challenges still open.
↗