Netflix has developed and maintains a large-scale knowledge graph system specifically designed for entertainment data, which is used across different teams and platforms within the organization. This case study presents an interesting evolution from traditional entity matching approaches to the incorporation of Large Language Models (LLMs) for enhancing graph capabilities and relationship inference. # System Overview and Traditional Approach The knowledge graph serves as a fundamental infrastructure component that connects various entities in the entertainment domain - movies, actors, countries, books, and semantic entities. The system's primary value comes from understanding and leveraging relationships between these entities, which is crucial for various use cases including: * Content similarity analysis * Search enhancement * Recommendation systems * Predictive analytics One of the core challenges in building and maintaining this knowledge graph is entity matching at scale. The system needs to accurately identify when two entities refer to the same thing (like different versions or references to the same movie) while avoiding false matches. This is particularly challenging with entertainment content where similar titles, remakes, and sequels are common. The team initially built a classification-based approach for entity matching that could handle billions of pairs of entities. They leveraged Metaflow's capabilities to create a highly parallelizable system that could: * Process large amounts of metadata efficiently using Apache Arrow * Distribute workloads across multiple nodes * Parallelize feature computation within each node * Write results in parallel without bottlenecks # LLM Integration and Modern Approaches The system has evolved to incorporate LLMs for more sophisticated relationship inference and entity type detection. The modern approach uses LLMs to: * Infer relationships that aren't explicitly stated in the data * Determine entity types and classifications * Extract information from unstructured content * Identify semantic relationships using the LLM's world knowledge The LLM integration pipeline follows a structured approach: 1. Takes input from the existing knowledge graph 2. Uses RAG (Retrieval-Augmented Generation) processes for querying 3. Employs specialized modules for entity type and relationship extraction 4. Supports multiple model types (rule-based NLP, LLM-based approaches) # Infrastructure and Operations The team has implemented several important operational features: * **Resource Management**: The system allows for fine-tuned resource allocation based on model complexity and requirements. This is particularly important when dealing with more resource-intensive LLM operations versus simpler rule-based approaches. * **Error Handling and Debugging**: Metaflow's capabilities allow for isolated debugging of failed instances, which is crucial when dealing with distributed systems at this scale. * **Version Control and Development Isolation**: The system implements strict separation between production and development environments through project versioning and branching models. This ensures that experimental work with new LLM models or approaches doesn't interfere with production systems. * **Monitoring and Optimization**: The team uses Metaflow's UI to monitor resource usage, detect skews in data processing, and identify bottlenecks in both traditional and LLM-based processing pipelines. # Technical Challenges and Solutions The implementation faced several technical challenges: * **Scale**: Processing billions of entity pairs efficiently required careful optimization of data parsing and processing pipelines. The team achieved a 10x speedup just in data parsing by using Apache Arrow and Metaflow's fast data layer. * **Accuracy**: Entertainment content presents unique challenges for entity matching due to remakes, sequels, and similar titles across different countries and languages. The system needs to maintain high accuracy while processing at scale. * **Resource Optimization**: Different models (from simple rule-based to complex neural networks to LLMs) require different resource allocations. The system needed to be flexible enough to accommodate these varying requirements while maintaining cost efficiency. # Future Directions The team is actively exploring expanded use of LLMs for: * More sophisticated relationship inference * Better understanding of content themes and subjects * Automatic franchise and sequel relationship detection * Enhanced semantic understanding of content relationships The modular nature of their Metaflow-based architecture allows them to experiment with different LLM approaches while maintaining production stability. This hybrid approach - combining traditional graph-based methods with modern LLM capabilities - represents a pragmatic path forward for large-scale knowledge graph systems. # Results and Impact While specific metrics weren't shared in the presentation, the system has demonstrated significant improvements in: * Processing speed (10x improvement in data parsing) * Scalability (handling billions of entity pairs) * Operational efficiency (better debugging and resource utilization) * Flexibility (supporting both traditional and LLM-based approaches) The system serves as a crucial infrastructure component at Netflix, supporting various teams and applications while continuously evolving to incorporate new technologies and approaches.