5 Things We Learned Building Ixorah in Production — Auxzon

The gap between "RAG works in a demo" and "RAG works reliably in production" is larger than most people expect. This is what we found. When we set out to build Ixorah, we wanted more than just a chatbot. We wanted a reliable, context-aware AI system that could act as the first point of contact for enterprise decision-makers. The journey from proof-of-concept to a production-ready business system taught us several hard truths about Retrieval-Augmented Generation (RAG). Here are five critical lessons from our own deployment. Lesson 1: Chunk size matters more than model choice Most people focus on which LLM to use. We found that how you split your knowledge base into chunks has a bigger impact on answer quality than the model itself. Too large, and irrelevant context confuses the model. Too small, and answers lack coherence. We went through 3 different chunking strategies before finding what worked for our specific knowledge base structure. The model is just the reasoning engine; your chunking strategy is the actual foundation of intelligence. Lesson 2: Retrieval quality is a moving target What retrieves well today may not retrieve well after you add 50 more documents. As we expanded our service descriptions, we found that certain embeddings started to overlap, causing the system to retrieve the wrong page for specific queries. We had to build evaluation pipelines — sets of test questions with expected answers — to catch quality regressions as the knowledge base grew. Without this, you won't know when retrieval silently degrades. Lesson 3: Users ask questions you never anticipated We indexed our knowledge base with Auxzon's service descriptions, FAQs, and process documents. Within the first week, users asked questions that fell completely outside any indexed content. Graceful fallback handling — knowing when NOT to answer versus when to answer confidently — is harder to build than the retrieval itself. A production RAG system must be explicitly instructed on its boundaries. Lesson 4: Latency is a UX problem, not just a performance problem Our first production version had acceptable latency for a developer testing it (~2-3 seconds). Real users in a chat context found it frustrating. We had to implement streaming responses and optimise our retrieval pipeline specifically for perceived speed, not just raw latency. Lesson 5: The knowledge base is a living system, not a one-time setup The hardest ongoing challenge isn't the AI — it's maintaining knowledge quality. Outdated documents in the knowledge base produce confidently wrong answers. We built internal processes for knowledge base review and update cycles before we could trust Ixorah in production. "The gap between 'RAG works in a demo' and 'RAG works reliably in production' is larger than most people expect." Would we do it again? Yes — but knowing what we know now, we'd spend the first month entirely on knowledge architecture and evaluation infrastructure before touching the retrieval or generation layer. Skip the 3-Month Learning Curve If you're evaluating building a RAG-powered AI system for your business, this is exactly the kind of problem we help companies navigate. Book a Free AI Audit Explore Knowledge AI