RAG Davinci

Executive summary

RAG Davinci is a backend service for Retrieval-Augmented Generation (RAG). It accepts documents in different formats, extracts and structures their content, creates vector embeddings, and answers questions using the most relevant document passages.

A useful RAG system needs more than an LLM call. It needs reliable document ingestion, background processing, retrieval-quality controls, source attribution, failure recovery, and operational visibility. RAG Davinci brings those concerns together behind a versioned REST API.

The problem

Teams often store useful knowledge across PDFs, office documents, scans, images, and text files. Conventional keyword search can miss semantic relationships, while sending entire documents to an LLM is expensive, slow, and limited by context windows.

The system therefore needed to:

• ingest heterogeneous files without blocking API requests;
• recover text from both digital and scanned documents;
• preserve page and document context for traceable answers;
• retrieve relevant passages using configurable strategies;
• keep queries available while new documents are processed; and
• expose job progress, health information, and retry paths.

What I built

Document ingestion

• Multi-file upload with file-type and size validation
• SHA-256-based duplicate detection
• Asynchronous processing through Celery and Redis
• Job lifecycle tracking across queued, processing, embedded, completed, and failed states
• Retry and document-reprocessing flows
• Local and Azure Blob Storage options

Parsing and OCR

• Parsers for PDF, DOCX, PPTX, XLSX, CSV, TXT, Markdown, and common image formats
• PyMuPDF as the lightweight PDF path and MinerU as an optional high-fidelity parser
• Tesseract OCR with image preprocessing
• Optional Azure Document Intelligence integration with OCR fallback behavior
• Page, section, table, and content-type metadata preservation where available

Retrieval and answer generation

• PostgreSQL with pgvector for durable vector storage
• Semantic chunking with configurable overlap and parent chunks
• Basic similarity search for predictable, lower-latency retrieval
• Optional MMR, hybrid search, adaptive thresholds, HyDE, query expansion, reranking, and neighboring-context expansion
• Search-only endpoints for retrieval debugging and evaluation
• Answers accompanied by source chunks, filenames, page numbers, and preview links

API and operations

• FastAPI REST service with OpenAPI documentation
• Versioned query APIs through /v1 and /v2
• API-key authentication for protected endpoints
• Optional in-memory rate limiting, centralized error handling, and request logging
• Health checks for the API, database, Redis, memory, and disk
• Docker Compose services for the API, worker, Redis, PostgreSQL, and optional Flower monitoring
• Document preview, page-content, and annotation endpoints

System architecture

The write path is asynchronous: the API validates an upload, stores its metadata, and queues work. A worker then parses, chunks, embeds, and persists the document.

Ingestion path

Query path

Files are retained through configurable local or Azure Blob Storage. Both the API and ingestion worker can access storage for upload processing, previews, and source references.

Main workflows

Ingestion

1. A client uploads one or more files.
2. The API validates the request and checks the file hash for duplicates.
3. A job and document record are created, then processing is delegated to a worker.
4. The worker selects an appropriate parser or OCR path.
5. Extracted content is divided into context-preserving chunks.
6. Embeddings and metadata are stored in PostgreSQL and pgvector.
7. Progress is updated throughout the job; recoverable failures can be retried.

Query

1. The client submits a question and optional document filters.
2. The question is embedded and used to retrieve candidate chunks.
3. The selected strategy may diversify, expand, or rerank the candidates.
4. The best passages become bounded context for the generation model.
5. The response includes the answer and traceable source references.

Technology stack

Engineering decisions

PostgreSQL plus pgvector instead of a separate vector database

Document metadata, ingestion state, annotations, and vectors share one transactional system. This reduces operational complexity and allows document deletion and filtering to remain consistent with vector records. A specialized vector database can still be introduced if scale or retrieval requirements justify it.

Asynchronous ingestion

OCR and high-fidelity PDF parsing can be CPU- and memory-intensive. Moving this work to Celery workers keeps the HTTP layer responsive, provides retry semantics, and lets API and worker resources scale independently.

Two retrieval API levels

The v1 API provides a stable semantic-search baseline. The v2 API exposes more expensive retrieval techniques as explicit options. This makes quality-versus-latency trade-offs measurable instead of applying every technique to every query.

Provider abstraction

Embedding, generation, OCR, parser, and storage choices are configured rather than embedded in endpoint logic. This supports local experimentation and cloud deployment without redesigning the core workflow.

Source-first responses

Every answer can return the passages that supported it, including the originating document and page where available. This does not eliminate hallucinations, but it makes responses easier to inspect, debug, and evaluate.

Reliability, security, and privacy

• Protected endpoints require an x-api-key when authentication is configured.
• Uploads are restricted by extension, file size, request count, and PDF page count.
• Secrets and provider credentials are supplied through environment variables.
• Database connections use pooling and pre-ping checks.
• Worker tasks report progress and preserve failure information for diagnosis.
• Health endpoints separate load-balancer checks from protected component diagnostics.
• Document deletion cascades to associated chunks and embeddings.

The service is intended for private document collections. A public or multi-tenant deployment would additionally require TLS at the edge, managed secrets, per-user authorization, audit logging, malware scanning, stricter network controls, and a defined privacy and retention policy.

Quality strategy

Current automated tests cover configuration behavior, text chunking, parser selection, text and Markdown parsing, and OCR routing. Search-only APIs make retrieval behavior inspectable without mixing it with answer generation.

The next quality milestone is a repeatable evaluation suite with a versioned question-and-answer dataset, retrieval metrics such as Recall@K and MRR, groundedness checks, latency percentiles, and regression thresholds. No benchmark numbers are published because a controlled public evaluation has not yet been completed.

Current limitations

• The API-key model is suitable for a controlled service, not full multi-tenant identity and access management.
• The optional in-memory rate limiter is not enabled by default and should become a distributed Redis-backed policy for multiple API replicas.
• OCR and parsing quality varies with document layout, scan quality, language, and selected provider.
• Advanced retrieval can improve difficult queries but adds latency and model cost.
• Horizontal scaling, backup and restore, and disaster-recovery procedures require deployment-specific validation.
• The project currently exposes a backend API rather than a polished end-user interface.

What this project demonstrates

• Designing an AI capability as an operable backend system rather than only a prototype prompt
• Building asynchronous, failure-aware document pipelines
• Working with vector search and advanced RAG retrieval patterns
• Integrating cloud and open-source AI providers behind configurable interfaces
• Modeling traceability through source metadata, page references, previews, and annotations
• Making explicit trade-offs among retrieval quality, latency, cost, and infrastructure complexity

Planned improvements

• Establish a reproducible RAG evaluation and regression pipeline
• Add tenant-aware authentication and authorization
• Move rate limiting and quotas to Redis
• Add OpenTelemetry traces and structured performance dashboards
• Improve deployment automation and database-migration workflows
• Build a small demonstration UI for upload, query, source preview, and annotation

Repository access

Private repository. The implementation contains ongoing experimental work and deployment-specific details. This case study documents the system for portfolio review without publishing credentials, private data, or proprietary configuration.

For a technical interview, I can walk through the architecture, explain the design trade-offs, demonstrate the API in a controlled environment, and discuss selected implementation details.