Enterprise AI Infrastructure & Tuning
Challenge: The organization required a secure, air-gapped AI environment to analyze sensitive documents and prevent external data leakage to public models. Crucially, the solution needed to serve multiple distinct departments (Legal, IT, HR) without cross-contamination of highly specialized knowledge.
Solution: Architected and deployed a massive on-premises AI server cluster for the entire enterprise using Docker, Ollama, and OpenWebUI. I custom-tuned an open-source 116.8B parameter model into several specialized variants, providing dedicated secure models for Legal, IT, and HR.
Key Features & Impact:
- Multimodal Vision: Integrated
LLaVAandLlama Vision, allowing the AI to ingest screenshots and documents across all departments for rapid contextual analysis and troubleshooting. - Specialized RAG & Routing: Deployed distinct document repositories for Lawyers, IT, and HR. The AI intelligently routes queries to the correct domain-specific repository or performs web searches, drastically increasing efficiency and eliminating AI hallucinations.
- Natural Voice AI: Implemented the
tts-1engine for fluid, human-sounding voice conversations with the assistant.