News Archive

Another harbinger of things to come sits on a hillside near the Hudson River in upstate New York, where an immersion lab with 15-foot walls and a 360-degree projection system transports Rensselaer Polytechnic Institute language students to China, virtually.

In a collaboration between Rensselaer and IBM Research, the Cognitive and Immersive Systems Laboratory (CISL) invites students interested in conducting research with us to attend an information session Wednesday, August 12th @ 2 pm online. RSVP to Maura Marsett <> or register on Handshake by 9am Wednesday, August 12th for WebEx session information.

The research conducted at CISL is currently focused on the following projects:

In a collaboration between Rensselaer and IBM Research, the Cognitive and Immersive Systems Laboratory (CISL) invites students interested in conducting research with us to attend an information session Thursday, April 30th @ 1:30 pm

Covid-19 Update: Following the state and university guidelines, HUMAINE organizers have decided to host this year's competition online via a web browser instead of running it in the on-campus immersive environment. At some future date, if conditions safely permit, we may demonstrate some of the submitted agents in the immersive environment. This does not affect how participants write/submit their code for now.

Following ANAC guidance, we have extended our deadlines for participants to submit their code.

Graduate Student Jerry Huang presents an ongoing CISL project at the 178th Meeting of the Acoustical Society of America in San Diego, California. 

link to webcast of Jerry's presentation

Assistant Professor Helen (Yalun) Zhou and Research Specialist Jaimie Drozdal host a roundtable discussion on the Mandarin Project at the American Council on the Teaching of Foreign Languages (ACTFL) 2019 Convention in Washington, D.C. 

The dream of perfect recreation of sound has always consisted of two parts: Reproduction of monaural sounds such that they seem to be exact copies of an original signal and the plausible recreation of complex sound environments, the possibility to be immersed in sound. The latter goal seems to be much more difficult, especially if we consider reproduction over headphones.

Universities have come a long way in their language studies, moving beyond just traditional classes in a lecture hall and prerecorded lessons without interaction. Now students have access to technologies such as artificial intelligence (AI) and natural language processing (NLP) in an immersive learning environment.

CVPR is the premier annual computer vision event comprising the main conference and several co-located workshops and short courses. With its high quality and low cost, it provides an exceptional value for students, academics and industry researchers. CVPR 2019 is hosted in Long Beach, CA. 3 papers came from Cognitive and Immersive Systems Lab (CISL) - the research collaboration between the IBM Research and Rensselaer Polytechnic Institute. CISL is one  research center in the AI Horizon Network (AIHN) university collaboration program.  

A student stands before a shopkeeper on a Chinese market street. The shopkeeper announces that he has many things to sell, from cake to postcards. The student, who is still learning how to master Mandarin, ponders the selection. Like many people learning a new language, he must consider grammar, pronunciation, and tenses before answering. When it comes to Mandarin—which the Foreign Service Institute of the U.S.

Often the best way to learn a language is to immerse yourself in an environment where people speak it. The constant exposure, along with the pressure to communicate, helps you swiftly pick up and practice new vocabulary. But not everyone gets the opportunity to live or study abroad.

A new proposal to the RPI KIP program was approved by KIP review committee in June, 2019 to support CISL research on Using Cognitive Computing for the Manufacturing Control Room of the Future. The new project was proposed by Prof. Wayne Bequette, Wayne Gray, James Malazita, David Mendonca, Rich Radke, and Hui Su. 

Brandyn Sigouin, first year CS undergraduate student and Research Assistant for the Cognitive and Immersive Systems Lab, wins an honorable mention for his Poster Presentation on Dialogue Engine for Teaching a Machine in the Computer and/or Computational Science category at the 2019 Undergraduate Research Symposium. 

2019 Undergraduate Research Symposium Poster and Oral Presentation Winners

CISL Director Hui Su and Staff members david allen and Andrea Wong presented the Rensselaer Mandarin Project this past week during Demo Session 1 of the Thirty-Third AAAI Conference on Artificial Intelligence (AAAI-19). 


Rensselaer Mandarin Project Demonstration

Recent advances in artificial intelligence, natural language processing, and mobile computing, together with the rising popularity of chat and messaging environments, have enabled a boom in the deployment of interactive systems based on conversation and dialogue. This talk explores the design and evaluation of conversational interfaces, and it is focused on design and evaluation methods which address specific challenges of interfaces based on multi-party dialogue. I will show two projects.

Out of 85 research posters from the AIHN community the Rensselaer Mandarin Project received one of six best poster awards at AI Research Week hosted by the MIT-IBM Watson AI Lab, in Cambridge, MA, October 1–5, 2018. 

All of the posters from the AI Horizons Colloquium are publicly available to review online. 


TROY — To find the powerful ancient manuscript, the student must first meet the mysterious Mrs. Ling — dressed in turquoise and with huge dark eyes that are a bit menacing — in an ornate tea house. And he must complete the Chinese tea ceremony correctly.

The Mandarin Project – a reboot of an initiative launched in 2012 to combine narrative, game design, and augmented and virtual reality to teach Chinese — is the latest manifestation of CISL, which is dedicated to pioneering immersive and cognitive systems as an aid to collaborative problem-solving.
Three CISL Collaborators — Jonas Braasch, School of Architecture, CISL; Mei Si, Cognitive Science Department, Games and Simulation Arts and Sciences (GSAS); and Yalun Zhou, Department of Communication and Media, Games and Simulation Arts and Sciences (GSAS), Mandarin Project — have been awarded a 2017 Rensselaer KIP grant to investigate new virtual reality methods to teach students pitch-related abilities including learning pitch contours for tone languages (Chinese) and correct intonation for musical instruments in conservatory-style education environments.
CISL Collaborators Professor Jonas Braasch and Samuel Chabot will present “An Immersive Virtual Environment for Congruent Audio-Visual Spatialized Data Sonifications” at the 23rd International Conference of Auditory Display at Penn State.