MediaRadar | Vivvix is an advertising intelligence company that helps marketers and agencies understand their competitive landscape by analyzing advertising content across various media channels. Their core challenge involved accurately classifying and extracting insights from video advertisements at scale, which initially relied heavily on manual processing by hundreds of operators. The case study demonstrates a comprehensive approach to implementing GenAI in a production environment, with several key technical components and considerations: ### Initial Challenges and Architecture The company initially struggled with a system built on Amazon SQS that had severe limitations, including manual polling requirements and a cap of 10 messages at a time. This created bottlenecks in meeting their SLAs. Their classification challenge was particularly complex, involving over 6 million unique products - what they termed "an extreme classification problem." The previous workflow was highly fragmented, with different components running in separate pods and limited visibility into system performance. Model deployment was cumbersome, requiring manual export and import procedures that consumed significant time and resources. ### Technical Implementation The new system architecture incorporates several sophisticated components: * Data Ingestion and Processing * Implemented Apache Spark Structured Streaming for continuous, real-time data ingestion * Developed preprocessing pipelines including video fingerprinting for duplicate detection * Integrated Whisper for transcription and OCR for text extraction from video frames * Built automated data transformation and monitoring capabilities * Model Architecture and Deployment * Utilized a dual-layer approach combining GenAI and traditional classification models * Deployed GPT-3.5 through Databricks Mosaic AI Model Serving for cost-effective processing * Implemented Ray clusters for distributed video processing * Created a hybrid system that compares GenAI outputs with their existing classification models to select the best matches * Monitoring and Governance * Consolidated monitoring across data sources, transformations, and model performance * Planned implementation of Unity Catalog for enhanced security and access control * Built comprehensive logging and performance tracking systems ### Production Considerations and Optimizations The team made several important decisions to optimize their production deployment: * Cost Management: They specifically chose GPT-3.5 over newer models to balance performance with operational costs, enabling them to process thousands of assets daily while maintaining reasonable expenses. * Performance Optimization: The system achieved significant performance improvements: * Increased classification throughput from 800 to 2,000 creatives per hour * Reduced model experimentation time from 2 days to 4 hours * Improved accuracy through the combination of multiple classification approaches * Operational Efficiency: The unified platform approach eliminated silos and provided better visibility into the entire pipeline, from data ingestion through model training and monitoring. ### Engineering Practices and Tooling The implementation demonstrates several LLMOps best practices: * Infrastructure Management * Use of containerized deployments for different components * Integration with cloud services (Azure) for scalability * Implementation of automated data and model pipelines * Development Workflow * Rapid prototyping capabilities through Mosaic AI Model Serving * Flexible development environment supporting both SQL and Python * Integrated monitoring and debugging tools * Security and Governance * Planned implementation of Unity Catalog for fine-grained access control * Data lineage tracking and monitoring * Secure model deployment and versioning ### Lessons and Insights The case study reveals several important insights for LLMOps implementations: * The importance of a unified platform approach to reduce operational complexity * The value of combining multiple AI approaches (GenAI + traditional models) for better accuracy * The need to balance model sophistication with operational costs * The benefits of automated preprocessing and data cleaning in video analysis * The importance of scalable infrastructure for handling increasing data volumes The implementation demonstrates a mature approach to LLMOps, showing how GenAI can be effectively integrated into existing workflows while maintaining performance, cost-effectiveness, and operational efficiency. The system's ability to handle increased load while reducing manual intervention showcases the potential of well-architected LLM-based systems in production environments.