Hugging Face's logo
Dan Saattrup Smart's picture
321 43 478

Dan Saattrup Smart PRO

saattrupdan
tintwotin's profile picture av1611's profile picture hannavang's profile picture
·
https://saattrupdan.com

AI & ML interests

NLP for low-resource languages.

Recent Activity

reacted to hannayukhymenko's post with ❤️ about 15 hours ago
Do you translate your benchmarks from English correctly? 🤔 Turns out, for many languages it is much harder than you can imagine! Introducing Recovered in Translation 🌍 together with @aalexandrov ritranslation.insait.ai Translating benchmarks is a painful process, requiring a lot of manual inspection and adjustments. You start from setting up the whole pipeline and adapting to every format type, including task specifics. There already exist some massive benchmarks, but they still have some simple (and sometimes silly) bugs, which can hurt the evaluations :( We present a novel automated translation framework to help with that! Eastern and Southern European languages introduce richer linguistic structures compared to English and for benchmarks which heavily rely on grammatical coherence machine translation presents a risk of harming evaluations. We discover potential answer leakage or misleading through grammatical structure of the questions. Some benchmarks are also just outdated and need to be retranslated with newer and better models. We present a framework with novel test-time scaling methods which allow to control time and cost investments, while at the same time mitigate the need for human-in-the-loop verification. While working on Ukrainian-focused MamayLM models, we had to translate 10+ benchmarks in a short span of time. Finding human evaluators is costly and time-consuming, same goes for using professional translators. With our pipeline we were able to do it in 3 days🏎️ We hope our findings will help enable stronger multilingual evaluations and developments. We release all produced benchmarks on Hugging Face together with the source code and Arxiv paper 🤗 Paper: https://huggingface.co/papers/2602.22207 Code: https://github.com/insait-institute/ritranslation Benchmarks: https://huggingface.co/collections/INSAIT-Institute/multilingual-benchmarks
liked a model 5 days ago
CoRal-project/roest-v3-chatterbox-350m
liked a model 5 days ago
CoRal-project/roest-v3-chatterbox-500m
View all activity

Organizations

Dansk Data Science Community's profile picture Flax Community's profile picture DaNLP's profile picture north's profile picture Blackbird AI's profile picture EuroEval's profile picture Alexandra Institute's profile picture Job Ad Generator's profile picture LumiOpen's profile picture Danish Foundation Models's profile picture CoRal's profile picture Merge Crew's profile picture RAG Demo's profile picture TrustLLM EU's profile picture European Values Project's profile picture Benchmark Translation Quality Estimation's profile picture A Smart Company's profile picture