AI in Usability Testing

Abstract

This thesis looks at how artificial intelligence can be used in usability testing. A custom API (Application Programming Interface) was built that works with GPT-4o to examine screenshots of user interfaces and check for usability problems. The tool was tested on four online unit converter websites. Its findings were then compared with feedback from students who reviewed the same websites as part of a class project. The study shows that AI can efficiently and consistently detect usability problems, especially in areas such as visibility, error prevention, and system feedback. At the same time, it highlights the value of human input. Students were able to offer personal insights and noticed details specific to each context that AI alone could not capture. The findings suggest that a hybrid approach, combining automated analysis with human judgment, offers a more complete usability testing framework. Future research should aim to improve AI transparency, address potential biases, and refine these tools for wider use in usability evaluation.