Comparing Human and AI Emotion and Sentiment Analysis in Mixed Reality and Zoom Environments

Abstract

While Mixed Reality devices and platforms have entered the digital market, the question of their effectiveness in providing remote presence remains unanswered. In this thesis, we aim to challenge two trending and rapidly growing technologies: Mixed Reality and Large Language Models (LLMs). By designing a case study focused on education—a key target for Mixed Reality—we aim to measure immersion using sentiment analysis based on Plutchik’s Wheel of Emotions. This analysis requires Finnish language skills since the study is conducted in Finland and in Finnish. Unlike our previous case study, which relied on translations by Finnish speakers, this research incorporates a pipeline that leverages LLMs to assist an English speaker in overcoming the language barrier. We then compare the results of the collaboration between an English speaker and AI against those of native Finnish speakers. Eventually, the Mixed Reality experience is categorized as an immersive experience, achieving a sentiment analysis rate of 0.75 for immersion. Additionally, the English speaker’s performance was found to be 19% less effective than that of the Finnish speakers and 24% less effective than using LLMs alone.