LLM Based Cross Modality Retrieval to Improve Recommendation Performance

Anwaar, Fahad; Khan, Adil Mehmood; Khalid, Muhammad

Abstract

The metadata of items and users play an important role in improving the decision-making process in the Recom-mender System. In recent times, web scraping-based techniques have been widely utilized to extract explicit user and item meta-data from different social platforms to improve recommendation performance. Currently, Large Language Models (LLMs) have the great potential to replace the traditional web scraping-based paradigm in Recommender Systems. In this paper, we investigated the impact of LLMs and web scraping-based extraction of explicit data on the performance of the Recommender System. Firstly, a cross-modality retrieval-based LLM Gemini is explored to generate semantically enriched textual descriptions of items from digital images. The Gemini LLM is prompted with few-shot prompting on the MovieLens dataset to generate a textual description of movies based on the corresponding movie poster. Secondly, the textual descriptions for each movie in the MovieLens dataset are scraped from the OMDB API. Finally, the cross-modality retrieval-based and scraping-based textual descriptions of items are incorporated into a hybrid Recommender System to assess the quality of explicit data in terms of recommendation performance. The experimental results on the MovieLens dataset demonstrate that LLM-generated content is more effective, achieving a 0.5134 RMSE in enhancing the performance of the Recommender System.

Citation

Anwaar, F., Khan, A. M., & Khalid, M. (2024, August). LLM Based Cross Modality Retrieval to Improve Recommendation Performance. Presented at 2024 29th International Conference on Automation and Computing (ICAC), Sunderland, UK

• Presentation Conference Type: Conference Paper (published)
• Conference Name: 2024 29th International Conference on Automation and Computing (ICAC)
• Start Date: Aug 29, 2024
• End Date: Aug 30, 2024
• Acceptance Date: Jun 8, 2024
• Online Publication Date: Oct 23, 2024
• Publication Date: Oct 23, 2024
• Deposit Date: Aug 30, 2024
• Publicly Available Date: Jan 14, 2025
• Publisher: Institute of Electrical and Electronics Engineers
• Peer Reviewed: Peer Reviewed
• Pages: 1-6
• DOI: https://doi.org/10.1109/ICAC61394.2024.10718796
• Keywords: Index Terms-Recommender System; LLMs; Web Scraping; NLP; Generative Content
• Public URL: https://hull-repository.worktribe.com/output/4793037