Experimenting with GPT OSS 20B and LoRA Fine-tuning to build Cooking Recipes GPT

Abstract

The evolution of Large Language Models has been fast. Even some recent smaller and open-weight models have proved to be capable of various tasks. One of them is GPT OSS 20B which turns out to be capable of producing versatile, quality cooking recipes out-of-the box. There exists also specialized cooking and recipes related datasets which could be used with these LLMs to fine-tune them. But experiments suggest that such fine-tuning is often not needed. GPT OSS 20B has gone through already extensive post-training optimization and tuning. Yet, the fine-tuning techniques have also been developing and become more accessible to more developers and institutions. LoRA as technique, is very useful and lightweight and still worth to try. In this thesis, LoRA is used to fine tune GPT-OSS 20B with cooking related conversational dataset. Another dataset consisting of recipes, is used to evaluate the understanding of the LLM of the cooking domain. The results show that GPT OSS 20B didn’t really benefit from such light fine-tuning but instead shines already on it’s own. One restriction was hardware which lead to using small amount of data, only affecting the style of the LLM. Using larger, high quality and versatile datasets is one thing which could be tested and studied in future research.