Skip to content
View olyamasaeva's full-sized avatar
1 follower · 2 following

Achievements

Achievement: YOLOAchievement: Pull Shark

Achievements

Achievement: YOLOAchievement: Pull Shark

Block or report olyamasaeva

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
olyamasaeva/README.md
Olga Masaeva — LLM Engineer · agentic systems · LLM evaluation

Email Telegram LinkedIn

I build agentic and RAG systems and check that they actually work. For three years I have shipped LLMs to production in banking, ads and EdTech: GigaChat search and assistants at Sber, self-hosted Qwen pipelines and a profession recommender serving Moscow's employment centre. Most of my work is evaluation: LLM judges, red-teaming and statistics that hold up on small samples.

Featured projects

paper-lab

paper-lab
A Claude Code skill that turns a paper into a Jupyter assignment you have to solve: the core of the method is a stub, the tests check you against the paper's numbers. Evaluated against Yandex School of Data Analysis homework with an LLM judge that separates real gaps from deliberate omissions (7.44/10 on 8 topics).

 book-translate

book-translate
A Claude Code skill that translates scanned books in place. Photos and sewing patterns stay uncovered; the output is an in-place PDF, a bilingual PDF and an offline reader with sentence-level alignment. Built on a 222-page Finnish book about folk costume.
ru-genealogy-search

ru-genealogy-search
An agent skill that finds ancestors in Russian and Soviet records that search engines miss: the notary inheritance registry, Leningrad evacuation lists, repression memory books, WWII losses. It builds a GEDCOM tree with a source and a confidence level for every fact.

Work

  • Sea of Data · LLM / AI Engineer · 2025–now. EdTech startup, AI career guidance for schools, universities and employment centres. Evaluation of a production AI tutor: a self-hosted Qwen judge scored 1,245 tutor turns on 18 pedagogical dimensions against 262 lessons of award-winning teachers. A profession recommender and learning-path builder in production for 3,600+ users, Moscow Employment Centre and Moscow Polytech.
  • Sber · LLM Engineer · 2024–2025. LLM-as-a-Judge evaluation for RAG that cut evaluation time by 35%; GigaChat query understanding for restaurant search; LoRA fine-tuning of GigaChat for marketing push notifications.
  • Yandex · Data Science Intern, Ads · 2023–2024. Ad ranking research, A/B tests, linear models in production.

Talks and writing

  • AI tutor and its evaluation methods · Data Fest 2026 · video
  • Automatic RAG evaluation systems review · Data Fest 2024 · video
  • Tune in to RAGAS and customize RAGAS for yourself · Habr, Sber blog, 2024 · article

Toolbox

Python · PyTorch · LangGraph · vLLM · Qwen · GigaChat · Claude Code · FastAPI · Docker · DuckDB · MLflow · DVC

ACM ICPC semi-finalist, informatics olympiad coach.

Pinned Loading

  1. Intellectual_parking_system Intellectual_parking_system Public

    Coursework, PetrSU, 2022 — Project of the course of Intellectual systems designing in PetrSU

  2. MADE__classic_ML MADE__classic_ML Public

    Coursework, VK Academy MADE, 2023 — Classic Machine learning course in MADE

    Jupyter Notebook

  3. Made_mlbd Made_mlbd Public

    Coursework, VK Academy MADE, 2022 — Made machine learning on big data course

  4. Made_MLOps Made_MLOps Public

    Forked from made-mlops-2022/Made-ML12-Masaeva-Olga

    Coursework, VK Academy MADE, 2023 — Machine Learning Operations system labs in MADE

    Jupyter Notebook

  5. TPPO TPPO Public

    Coursework, PetrSU, 2020 — Course project of measure analysis in PetrSU

    Jupyter Notebook

  6. Signal-Analysis Signal-Analysis Public

    Coursework, PetrSU, 2022 — Signal analysis labs in PetrSU

    Jupyter Notebook