FLANS at SemEval-2026 task 7: RAG with open-sourced smaller LLMs for everyday knowledge across diverse languages and cultures

Bogdanova, L.; Sun, S.; Han, L.; Amat Lefort, N.; Plaza del Arco, F.M.; Kochmar, E.; Ghosh, D.; North, K.; Komachi, M.; Zampieri, M.

doi:doi:10.18653/v1/2026.semeval-1.119

Persistent URL of this record https://hdl.handle.net/1887/4309144

View statistics

In Collections

This item can be found in the following collections:

Faculty of Science: Other

Bogdanova, L.; Sun, S.; Han, L.; Amat Lefort, N.; Plaza del Arco, F.M. (2026)

FLANS at SemEval-2026 task 7: RAG with open-sourced smaller LLMs for everyday knowledge across diverse languages and cultures

Article in monograph or in proceedings

This system paper describes our participation in the SemEval-2025 Task-7 “Everyday Knowledge Across Diverse Languages and Cultures”. We attended two subtasks, i.e., Track 1: Short Answer Questions (SAQ), and Track 2: Multiple-Choice Questions (MCQ).The methods we used are retrieval augmented generation (RAGs) with open-sourced smaller LLMs (OS-sLLMs). To better adapt to this shared task, we created our own culturally aware knowledge base (CulKBs) by extracting Wikipedia content using keyword lists we prepared. We extracted both culturally-aware wiki-text and country-specific wiki-summary. In addition to the local CulKBs, we also have one system integrating live online search output via DuckDuckGo.Towards better privacy and sustainability, we aimed to deploy smaller LLMs (sLLMs) that are open-sourced on the Ollama platform.We share the prompts we developed using refinement techniques and report the learning curve of such prompts.The tested languages are English, Spanish, and...Show moreThis system paper describes our participation in the SemEval-2025 Task-7 “Everyday Knowledge Across Diverse Languages and Cultures”. We attended two subtasks, i.e., Track 1: Short Answer Questions (SAQ), and Track 2: Multiple-Choice Questions (MCQ).The methods we used are retrieval augmented generation (RAGs) with open-sourced smaller LLMs (OS-sLLMs). To better adapt to this shared task, we created our own culturally aware knowledge base (CulKBs) by extracting Wikipedia content using keyword lists we prepared. We extracted both culturally-aware wiki-text and country-specific wiki-summary. In addition to the local CulKBs, we also have one system integrating live online search output via DuckDuckGo.Towards better privacy and sustainability, we aimed to deploy smaller LLMs (sLLMs) that are open-sourced on the Ollama platform.We share the prompts we developed using refinement techniques and report the learning curve of such prompts.The tested languages are English, Spanish, and Chinese for both tracks.Our resources and codes are shared via \url{https://github.com/aaronlifenghan/FLANS-2026}Show less

RAG

Diverse languages and cultures

NLP

LLM