AI Customer Support Chatbot
RAG-based chatbot that resolves customer complaints by retrieving similar past cases and generating responses with a local LLM.
Overview
A customer support chatbot powered by Retrieval-Augmented Generation (RAG). The system embeds historical complaint-solution pairs into a vector database and retrieves the most similar past cases to inform LLM-generated responses — all running locally without cloud API costs.
Problem Statement
Traditional rule-based chatbots fail to handle the natural language variability of real customer complaints. Large cloud LLMs are expensive and raise privacy concerns for sensitive support data. A local RAG pipeline offers the best of both worlds.
Architecture
- FastAPI backend for REST API endpoints
- PostgreSQL with pgvector extension for vector similarity search
- Sentence-transformers for generating high-quality text embeddings
- Ollama running the 'phi' model locally for response generation
- RAG pipeline: embed query → retrieve top-k similar past issues → generate response
Challenges
- ✗Running a capable LLM locally with acceptable latency
- ✗Storing and querying vector embeddings efficiently in PostgreSQL
- ✗Designing prompts that ground the LLM to retrieved context
Solutions
- ✓Used Ollama with the phi model — fast and lightweight for local inference
- ✓pgvector enables native vector search without a separate vector database
- ✓Carefully engineered system prompts to prevent hallucination
Lessons Learned
- 💡RAG is highly effective for domain-specific, knowledge-grounded chatbots
- 💡pgvector is a great choice when you already have PostgreSQL in your stack
- 💡Local LLMs like phi are surprisingly capable for structured support tasks
Related Projects
Brain CT Scan Classifier
Multi-label classification system for brain CT scans using ConvNeXt-V2 and Multiple Instance Learning with a dual-head architecture.
Microjobs Portal
A local part-time job board with an AI-powered recommendation system using TF-IDF and FAISS for fast similarity search.
Snake Maramari — Multi-Snake AI Battle
A competitive snake game in Python and Pygame where up to 4 AI snakes with different algorithms fight for survival and steal scores.