# Whatnot's LLM Implementation for Trust and Safety Whatnot, a rapidly growing live shopping marketplace, has implemented an innovative approach to trust and safety using Large Language Models (LLMs). This case study details their journey from a traditional rule-based system to an advanced LLM-powered solution that handles content moderation, fraud detection, and policy enforcement. # System Evolution and Architecture ## Initial Rule Engine - Started with a centralized rule engine as the foundation - Focused on data-driven policy enforcement - Processed multiple data sources for violation detection - Handled routine tasks like shipping delays, refunds, and cancellations - Limited to scalar values and struggled with contextual understanding ## Enhanced System with LLMs - Integrated LLMs to overcome rule engine limitations - Created a three-phase architecture: # Technical Implementation Details ## Scam Detection System - Implements contextual analysis of user interactions - Uses multiple signals to trigger LLM analysis: ## Prompt Engineering - Carefully crafted prompts for scam detection - Structured JSON output format: - Example prompt structure includes: ## Integration with Existing Systems - LLM outputs feed back into rule engine - Decision matrix combines multiple factors: - Uses Kafka for system updates and product access changes # Advanced Features and Adaptations ## Multimodal Processing - Implemented OCR for analyzing text in images - Handles evolving fraud tactics - Adapts to different messaging patterns - Expanded to cover multiple policy violations: ## Machine Learning Approach - Utilizes zero-shot and few-shot learning - Plans for fine-tuning in specific use cases - Maintains human-in-the-loop verification - Combines AI insights with human judgment # System Performance and Results ## Metrics and Achievements - 95% detection rate for scam attempts - 96% precision in fraud detection - Rapid detection within minutes of attempt - High recall rate in identifying violations ## Operational Implementation - Three-tier enforcement system: - Automated notification system for violations - Dynamic product access modifications # Technical Stack and Integration ## Trust and Safety LLM Stack - Modular architecture for different violation types - Scalable processing pipeline - Integration with existing ML models - Real-time data processing capabilities ## Data Processing Pipeline - Event data collection - User data integration - Order history analysis - ML model output incorporation # Future Developments ## Planned Improvements - Moving towards unified Gen AI system - Enhanced fine-tuning for support cases - Expansion to new use cases - Stronger integration between rule engine and enforcement The system demonstrates a sophisticated approach to LLMOps in production, combining traditional rule-based systems with modern LLM capabilities. The architecture maintains flexibility while ensuring robust policy enforcement, showing how LLMs can be effectively deployed in critical trust and safety applications while maintaining human oversight and control.