FactSet is a leading provider of financial data and analytics solutions that serves various segments of the financial industry, including buy-side, sell-side, wealth managers, and private equity firms. Their case study presents a comprehensive journey of implementing a standardized LLMOps framework for enterprise-wide GenAI applications. The company's initial approach to GenAI implementation was characterized by decentralized development practices, where different teams used various tools and environments. This led to several operational challenges including collaboration barriers, duplicated efforts, and inconsistent quality across applications. Teams were working in silos, using different tools ranging from cloud-native commercial offerings to on-premises solutions, making it difficult to maintain standards and share resources effectively. To address these challenges, FactSet implemented a comprehensive LLMOps framework based on Databricks Mosaic AI and MLflow. The solution addressed several key areas: Data Management and Preparation: The framework leverages Delta Live Tables for data ingestion and transformation, allowing teams to create reusable pipelines for various use cases. For example, they implemented an end-to-end pipeline for earnings call summarization that includes data ingestion, text chunking, embedding generation, and vector search index updates. This standardized approach to data preparation has significantly improved the efficiency of their RAG applications. Governance and Lineage: A crucial aspect of their implementation was the use of Unity Catalog, which solved previous challenges related to data silos and governance. The system provides hierarchical structure and fine-grained governance of data, models, and assets. It enables isolation at both metadata and physical storage levels, allowing different teams to work in their own spaces while maintaining centralized oversight. The platform captures table/column level lineage for all operations, which is crucial for monitoring and explaining downstream GenAI applications. Model Development and Deployment: The framework supports both commercial and open-source models, allowing teams to choose the best option for their specific use cases. They implemented MLflow for model experimentation, versioning, and deployment, which has streamlined the development process and enabled better collaboration across teams. The MLflow Deployments Server provides simplified model serving for various model types. Practical Applications and Results: The case study highlights two significant applications that demonstrated the value of their LLMOps framework: 1. Code Generation Component (Mercury): Initially using a commercial model that had high latency issues (over one minute response time), they successfully fine-tuned meta-llama-3-70b and DBRX models, achieving a 70% reduction in average user request latency. This improvement significantly enhanced the user experience in FactSet workstations. 2. Text-to-Formula Generation: They developed a sophisticated RAG workflow for generating FactSet formulas from natural language queries. The initial implementation hit accuracy limitations, but through their new framework, they were able to implement a compound AI architecture that improved both accuracy and latency. The end-to-end latency was reduced by approximately 60% through the use of fine-tuned open-source models. Cost Optimization and Model Selection: The framework enables careful evaluation of model costs and performance. Teams can now choose between commercial and open-source models based on specific use case requirements, leading to more cost-effective solutions while maintaining high performance standards. Their analysis includes detailed cost comparisons for different model types and use cases. Integration with Internal Systems: FactSet integrated the Databricks platform with their internal GenAI Hub, which manages all ML and LLM resources across projects. This integration enables centralized management of workspaces, model catalogs, and other essential metadata, facilitating collaboration between ML producers and consumers. They also implemented cost attribution workflows using Databricks cost views for better business transparency. Key Technical Implementations: - Vector search capabilities for efficient RAG applications - Fine-tuning pipelines for various open-source models - Automated data ingestion and transformation workflows - Centralized model serving infrastructure - Comprehensive monitoring and evaluation systems The framework has successfully democratized AI workflows that were traditionally limited to specialized AI engineers. It provides a balanced approach that maintains flexibility while enforcing standardization and best practices. The platform supports their strategy of using the right model for each specific use case, whether it's a fine-tuned open-source model or a commercial LLM. Impact on Development Culture: The standardized framework has fostered a culture of collaboration and innovation while maintaining governance and control. Development teams can now focus on building solutions rather than managing infrastructure, leading to faster development cycles and more consistent quality across applications. FactSet's case study demonstrates how a well-designed LLMOps framework can transform an organization's approach to AI development, enabling both innovation and standardization while maintaining necessary controls and governance. Their success in reducing latency, improving accuracy, and optimizing costs provides a valuable blueprint for other organizations looking to implement enterprise-wide GenAI solutions.