# Building Reliable Agentic Systems at Factory.ai Factory.ai is building a platform to bring autonomy to software engineering, enabling AI systems to collaborate with developers on various tasks from coding to documentation and code review. This case study examines their practical experiences and approaches to building reliable AI agent systems in production. ## Core Agent Characteristics Factory.ai defines agentic systems through three key characteristics: - Planning capabilities - Making plans for future actions - Decision-making abilities - Evaluating data and selecting actions - Environmental grounding - Reading and writing to external environments ## Planning Reliability Techniques ### Context Propagation - Inspired by Kalman filters from robotics - Passes intermediate context through multi-step plans - Helps maintain focus and convergence - Trade-off: Initial errors can propagate through steps ### Subtask Decomposition - Breaks complex plans into smaller manageable tasks - Enables finer control over execution - Risk: Potential for introducing incorrect steps - Requires careful balance between granularity and reliability ### Model Predictive Control - Dynamic plan adaptation based on real-time feedback - Allows agents to adjust to problems encountered - Challenge: Higher risk of straying from intended trajectory - Requires mechanisms to balance adaptation vs consistency ### Plan Success Criteria - Clear definition of plan structure and success metrics - Can use instruction prompts, few-shot examples, type checking - Helps maintain consistency and reliability - Difficulty scales with complexity of problem space ## Decision-Making Improvements ### Consensus Mechanisms - Samples multiple outputs to improve accuracy - Techniques include: - Trade-off: Higher inference costs ### Explicit Reasoning Strategies - Uses techniques like Chain of Thought and checklists - Reduces complexity of decision processes - Improves consistency but may limit creativity - Success depends on choosing appropriate reasoning frameworks ### Model Weight Optimization - Fine-tuning for specific decision tasks - Effective for known task distributions - Expensive and locks in quality level - Often base models perform better due to generality ### Simulation-Based Decisions - Samples and simulates multiple decision paths - Uses reward criteria to determine optimal choices - Similar to Monte Carlo tree search - Challenges: ## Environmental Grounding ### Tool Design - Build dedicated interfaces for agent-system interaction - Consider appropriate abstraction layers - Balance between capability and reliability - Examples range from simple calculators to sandboxed code execution - Challenge: Maintenance overhead for specialized tools ### Feedback Processing - Critical for handling complex environment data - Requires careful preprocessing of inputs - Example: Processing large volumes of log data - Focus on extracting relevant signals from noise ### Bounded Exploration - Enable controlled information gathering - Balance between exploration and task focus - Helps with understanding problem context - Risk of context overload or path deviation ### Human Integration - Design careful human-agent interaction points - Plan for human oversight and intervention - Recognition that 100% automation rarely achievable - Focus on effective human-AI collaboration ## Production Considerations - Reliability at scale requires multiple complementary approaches - Important to choose appropriate techniques for specific use cases - System design should account for both automation and human interaction - Testing and monitoring crucial for production deployment ## Lessons Learned - Tool interfaces are key differentiating factors - Base models often sufficient vs fine-tuning - Simpler, more constrained tasks easier to make reliable - Human oversight remains important even with advanced capabilities - Balance needed between flexibility and reliability ## Technical Architecture Decisions - Modular approach to agent capabilities - Focus on robust tool interfaces - Clear boundaries for agent operations - Multiple fallback mechanisms - Explicit success criteria and evaluation - Careful management of model inputs and outputs