Language Learning in a Cognitive and Immersive Environment Using Contextualized Panoramic Imagery

Immersive technologies augmented with AI for foreign language learning are currently uncommon. The Rensselaer Mandarin Project is a cognitive and immersive classroom that facilitates cultural and foreign language learning beyond a traditional classroom setting. Students perform tasks through intelligent interactions including authentic dialogues with multiple AI agents and multimodal gestures. Street View technology has the potential to further enhance foreign language learning and cultural knowledge acquisition in the Mandarin Project. Real-world panoramic scenes immerse learners in new locations and can provide richer context for more meaningful language learning. Students embark on “field trips” with cultural, historical, and lingual relevance at human-scale. Multiple learning opportunities and interactions are constructed for the various types of scenes students visit: vocabulary games, relevant AI agent dialogues, and informational lessons on cultural elements. The Mandarin Project will be assessed during a six-week classroom experience in the summer of 2019.