AI Chatbot

RAG AI Knowledge Base & Document Chatbot

An end-to-end RAG system that turns a private document library into a chatbot with grounded, source-cited answers.

Discuss your project

Documents feeding a RAG pipeline into a source-cited chatbot

ClientRAG AI Knowledge Base & Document Chatbot

ProjectAI Chatbot

Delivered byDevFlips

The project

About RAG AI Knowledge Base & Document Chatbot

Built an end-to-end Retrieval-Augmented Generation (RAG) system in Python for querying a private document knowledge base. The system ingests structured Markdown documents, splits them into meaningful semantic chunks by heading, paragraph, list, table and code block, and converts each chunk into a 384-dimensional embedding stored in Qdrant with its source file, chunk number and heading path. A user's question is embedded and matched against the closest chunks, which are passed to a local Ollama/Gemma LLM to generate a grounded answer — with source transparency and multi-turn conversation history.

Core workflow: documents → semantic chunking → embeddings → Qdrant → semantic retrieval → local LLM → grounded answer. Built with Python, Sentence Transformers, Qdrant, Docker, Ollama and Gemma.

The challenge

Answering questions from private documents, accurately

Generic chatbots either don't know a business's own documents, or make something up when they're unsure. The system needed to answer only from the real source material — and show where each answer came from.

The approach

Retrieval before generation

Split documents into semantic chunks, embed them, and store them in a vector database. At query time, retrieve only the most relevant chunks and pass them to a local LLM — so answers are grounded in real content, not guesses.

The solution

A closer look at how it works

Semantic chunking

Documents are split by heading, paragraph, list, table and code block — not a fixed character count — so each chunk stays meaningful on its own.

Source-cited answers

Every answer links back to the source file and heading it came from, so nothing is a black box.

Runs on a local LLM

Ollama and Gemma handle generation locally, so the knowledge base doesn't depend on sending data to a third-party API.

Results

What it made possible

Visitors' questions get answered from the real document set, with sources attached
New documents can be added without retraining anything — just re-embedding
Multi-turn conversation history keeps follow-up questions in context

More work