Conversational UI in browser extensions with agentic AI

Abstract

Conversational user interfaces have increased in popularity in the form of virtual assistants like Siri, and also chatbots in customer service settings. The emergence of large language models has improved the technology of natural language processing and enables the possibility of a conversational user interface capable of operating in the browser utilising LLMs with a browser extension. As there is no scientific research on such conversational browser-using agent browser extensions, the thesis gives an overview on the currently existing extensions and tests for their feasibility with an empirical study. A taxonomy for evaluating the extensions is created, and the empirical study uses different tasks to judge the capabilities of three extensions. The results of the study showcase that some of the existing extensions have very promising capabilities in operating in diverse web environments, and are able to operate based on vague natural language instructions with minimal oversight. The biggest challenge they face is their relatively slow execution speed, with multi-layered tasks and the correct choice of LLM and its instructions being among other challenges.