# Building and Deploying AI Agents at Rasgo Rasgo has developed a platform that enables customers to utilize AI agents for data analysis tasks, specifically focusing on database interactions and custom agent creation. This case study provides valuable insights into the practical challenges and solutions in deploying LLMs in production environments. # Core Technical Architecture ## Agent System Design - Agents are built with minimal code requirements: - System prompts are structured with clear sections: ## Technical Infrastructure - Database connectors for major platforms (Snowflake, BigQuery) - Security implementation with OAuth and SSO integration - Tool calls for: # Key LLMOps Learnings ## Model Selection and Management - Found GPT-4 significantly outperformed GPT-3.5 for complex reasoning tasks - Decision making capabilities of the base model heavily impact agent performance - Avoided fine-tuning for general agent behavior - Recommendation to support multiple model providers for flexibility ## Agent-Computer Interface (ACI) Design - Critical importance of tool call syntax and structure - Different models require different ACI approaches - Examples of ACI optimization: ## Production Infrastructure Requirements - Security implementation: - Data connector management: - User interface considerations: - Memory management: ## Evaluation and Testing - Non-deterministic nature requires comprehensive testing: - Test case development: ## Technical Recommendations - Vector search implementation: - Cost optimization: - Token streaming implementation for latency management # Production Challenges and Solutions ## Error Handling - SQL query failures require robust error handling - Return detailed error context to agents - Enable agents to self-correct through iteration ## Performance Optimization - Focus on reasoning capabilities over knowledge base size - Implement proper context retrieval systems - Balance token usage with response quality ## Security Considerations - Agent access control implementation - User permission management - Data access restrictions - Secure token handling # Best Practices and Recommendations ## Development Approach - Avoid unnecessary abstractions (like LangChain, LlamaIndex) - Own the model interaction layer - Maintain flexibility for model switching - Focus on production-ready infrastructure ## Monitoring and Maintenance - Track agent performance metrics - Monitor tool call success rates - Log decision paths and reasoning - Regular evaluation of model performance ## Scaling Considerations - User onboarding processes - Debugging capabilities - Performance monitoring - Version upgrade management The case study emphasizes that successful LLM deployment requires much more than just building capable agents. The focus on infrastructure, security, and user experience aspects highlights the complexity of running AI systems in production environments. Rasgo's experience shows that careful attention to the agent-computer interface, proper model selection, and robust infrastructure are critical success factors in deploying LLM-based systems.