Language Learning in a Cognitive and Immersive Environment Using Contextualized Panoramic Imagery

Immersive technologies augmented with AI for foreign language learning are currently uncommon. The Rensselaer Mandarin Project is a cognitive and immersive classroom that facilitates cultural and foreign language learning beyond a traditional classroom setting. Students perform tasks through intelligent interactions including authentic dialogues with multiple AI agents and multimodal gestures. Street View technology has the potential to further enhance foreign language learning and cultural knowledge acquisition in the Mandarin Project. Real-world panoramic scenes immerse learners in new locations and can provide richer context for more meaningful language learning. Students embark on “field trips” with cultural, historical, and lingual relevance at human-scale. Multiple learning opportunities and interactions are constructed for the various types of scenes students visit: vocabulary games, relevant AI agent dialogues, and informational lessons on cultural elements. The Mandarin Project will be assessed during a six-week classroom experience in the summer of 2019.

Reference

Samuel Chabot, Jaimie Drozdal, Yalun Zhou, Hui Su, and Jonas Braasch, "Language Learning in a Cognitive and Immersive Environment Using Contextualized Panoramic Imagery,"

In: Stephanidis C. (eds) HCI International 2019 - Posters. HCII 2019. Communications in Computer and Information Science, vol 1034. Springer, Cham

Bibtex

@InProceedings{10.1007/978-3-030-23525-3_26,
author="Chabot, Samuel
and Drozdal, Jaimie
and Zhou, Yalun
and Su, Hui
and Braasch, Jonas",
editor="Stephanidis, Constantine",
title="Language Learning in a Cognitive and Immersive Environment Using Contextualized Panoramic Imagery",
booktitle="HCI International 2019 - Posters",
year="2019",
publisher="Springer International Publishing",
address="Cham",
pages="202--209",
isbn="978-3-030-23525-3"
}