PROJECT / 01Applied AI · Retrieval · Evaluation

A document-grounded RAG system at OeNB

A useful RAG system should make it obvious what supports an answer—and know when the evidence is not enough.

Role
AI Engineer Intern
Context
Oesterreichische Nationalbank (OeNB) · Vienna
Period
Jul 2024 — Mar 2025
INTERACTIVE EVIDENCE

Explore the full pipeline.

A document-grounded AI workflow that treats retrieval, provenance, evaluation, and refusal as parts of one system.

Sanitized technical reconstruction
System pipelineSelect a stage to inspect it
STAGE / RETRIEVE

Retrieve

Compare the question with the indexed collection, rank candidates, and evaluate retrieval behaviour with recall and NDCG.

Input
Question representation
Output
Ranked candidate passages
  • Cosine similarity
  • Recall
  • NDCG
Synthetic request traceEvidence boundary passed
Which expenses require a second approval?
queryretrieveguardanswer
Answer generated with supporting passagesThe supporting passages stay attached to the resulting answer.
THE QUESTION

Turn mixed-format institutional documents into reliable context for language models while keeping answers inspectable and unsafe or unrelated requests outside the system boundary.

HOW I APPROACHED IT
  1. 01

    Built ingestion and preprocessing paths for PDF, TXT, and DOCX sources.

  2. 02

    Connected retrieval over PostgreSQL-backed document representations to Hugging Face and vLLM-based language-model workflows.

  3. 03

    Evaluated retrieval with recall and NDCG, and explored structured prompting, reflection, few-shot examples, and fine-tuning.

  4. 04

    Applied cosine-similarity guardrails and built tooling that described graphs for more accessible document use.

FormatsPDF · TXT · DOCX
RetrievalRecall · NDCG
BoundarySimilarity guardrail
BOUNDARY / LIMITATION

This page is a sanitized mechanism reconstruction. It uses synthetic documents and contains no internal OeNB data, screenshots, production measurements, or confidential implementation details.

TOOLS & METHODS
  • Python
  • PostgreSQL
  • Hugging Face
  • vLLM
  • NLTK
  • Stanza
  • Containers