# LLM-Powered Data Governance at Grab: Metasense V2 ## Overview Grab, a leading Southeast Asian superapp platform, developed Metasense V2 as an enhancement to their LLM-powered data governance system. The system was designed to automate governance-related metadata generation and data classification tasks across their extensive data lake. This case study details their journey in improving and productionizing their LLM system, focusing on technical implementations and operational improvements. ## Initial Implementation and Scale - The initial system was deployed to handle column-level tag classifications for determining data sensitivity tiers - System processed over 20,000 data entries at a rate of 300-400 entities per day - Deployment covered the entire data lake, significantly reducing manual classification workload ## Technical Challenges and Solutions ### Model Capacity Optimization - Identified key challenges in model performance: - Implemented solutions to address model capacity issues: ### LLMOps Infrastructure Improvements - Integrated LangChain Framework: - LangSmith Integration: ## Operational Improvements ### Prompt Engineering and Management - Developed streamlined prompt optimization workflow: ### Quality Assurance and Monitoring - Implemented robust quality control measures: ### Collaboration and Development Workflow - Enhanced team collaboration through: ## Performance Metrics and Results - Achieved significant improvements in classification accuracy - Maintained low misclassification rates - Reduced manual review requirements - Enhanced trust in model outputs from table owners ## Technical Implementation Details ### Architecture Components - LangChain Framework: - LangSmith Platform: ### Safety and Monitoring - Implemented comprehensive safety measures: ## Best Practices and Lessons Learned ### Development Process - Importance of iterative prompt optimization - Value of splitting complex tasks into manageable components - Benefits of robust monitoring and evaluation systems ### Team Collaboration - Cross-functional team enablement - Streamlined development workflows - Unified platform for collaboration ### Quality Assurance - Regular performance monitoring - Automated safety measures - Continuous improvement protocols ## Future Considerations - Continuous model refinement - Expansion of classification capabilities - Further automation of review processes - Enhanced monitoring systems ## Impact and Benefits - Significantly reduced manual workload - Improved classification accuracy - Enhanced team collaboration - Streamlined development process - Robust quality assurance - Scalable solution for data governance The case study demonstrates the successful implementation of LLMOps best practices in a production environment, highlighting the importance of careful system design, robust monitoring, and continuous improvement in maintaining high-quality LLM-powered services.