Back to Projects

RAG Platform

Enterprise AI Solution

Configurable Retrieval-Augmented Generation platform for deploying Q&A services on domain-specific documents.

Overview

A comprehensive enterprise RAG platform designed to enable domain-specific question-and-answer capabilities across AT&T. The platform provides a configurable architecture that allows teams to deploy custom Q&A services on their proprietary documents while maintaining strict data privacy and security standards.

Challenges

  • Handling diverse document formats and structures across different business units
  • Ensuring low-latency responses for real-time user queries at enterprise scale
  • Maintaining data privacy compliance while leveraging cloud-based LLM services
  • Optimizing embedding model performance for domain-specific terminology
  • Managing vector store scalability for millions of document chunks

Solutions

  • Implemented modular document processing pipeline supporting PDF, Word, HTML, and custom formats
  • Deployed NVIDIA Triton inference server for optimized model serving with batching and caching
  • Created hybrid architecture with on-premise embedding models and secure API gateways
  • Fine-tuned embedding models on AT&T-specific terminology and documentation
  • Utilized Milvus vector database with horizontal scaling and Azure Cognitive Search for hybrid retrieval

Key Results

Reduced document search time from hours to seconds for support teams
Achieved 95% accuracy on domain-specific queries after fine-tuning
Successfully deployed across 5+ business units with 10,000+ daily queries
Maintained 99.9% uptime with sub-second response times
Enabled self-service knowledge access, reducing support ticket volume by 40%

Technologies

OpenAIMilvusAzure SearchKubernetesPythonFastAPI