Company
Farfetch
Title
Multimodal Search and Conversational AI for Fashion E-commerce Catalog
Industry
E-commerce
Year
2023
Summary (short)
Farfetch developed a multimodal conversational search system called iFetch to enhance customer product discovery in their fashion marketplace. The system combines textual and visual search capabilities using advanced embedding models and CLIP-based multimodal representations, with specific adaptations for the fashion domain. They implemented semantic search strategies and extended CLIP with taxonomic information and label relaxation techniques to improve retrieval accuracy, particularly focusing on handling brand-specific queries and maintaining context in conversational interactions.
Farfetch, a leading online fashion marketplace, presents a comprehensive case study on implementing and deploying a multimodal conversational AI system called iFetch. This case study provides valuable insights into the practical challenges and solutions for deploying AI systems in a production e-commerce environment. The primary business motivation stems from customer behavior analysis showing that increased product exploration and "adds to bag" correlate strongly with conversion rates. The company identified key customer experience priorities: speed, convenience, and thoughtful service, based on market research. This guided their technical implementation strategy, focusing on practical improvements rather than just adopting cutting-edge technology for its own sake. The technical implementation follows a well-structured, incremental approach to building a production-ready system: **Data and Training Strategy** * The team leveraged existing customer query data from their platform * They augmented this with synthetic data generation, including: * Introducing deliberate grammar errors * Removing stop words * Creating random token permutations * This approach helped create a more robust training dataset that better reflects real-world usage patterns **Architectural Decisions** The system architecture demonstrates several important LLMOps considerations: * They chose semantic search over traditional lexical matching to handle language nuances better * The implementation uses vector databases for efficient retrieval (though specific details are mentioned as out of scope) * The system maintains conversational context across interactions, enabling follow-up queries **Multimodal Implementation** The team's approach to multimodal search showcases several sophisticated LLMOps practices: * They started with text-only capabilities before expanding to multimodal features, demonstrating a practical incremental deployment strategy * They adapted CLIP (Contrastive Language-Image Pre-training) for their specific use case * Key modifications to CLIP included: * Integration of fashion-specific taxonomic information * Implementation of a relaxed contrastive loss function * Specific handling for brand-related queries * Custom data augmentation techniques **Production Challenges and Solutions** The case study reveals several important production considerations: * Brand-specific query handling initially showed poor performance, requiring specific optimization * The team implemented specialized training strategies to handle different text input combinations * They maintained taxonomic relevance while improving brand and color-specific results * The system needed to handle both product listing pages (PLP) and product display pages (PDP) seamlessly **Evaluation and Monitoring** The team implemented several evaluation strategies: * User behavior tracking across the conversion funnel * Performance monitoring for different query types (text, visual, and combined) * Small group user testing for feedback and iteration * Specific attention to brand-related query performance **Integration with Existing Systems** The implementation shows careful consideration of integration with existing e-commerce infrastructure: * Connection with product catalog systems * Integration with existing user interface elements * Handling of product metadata and image data * Maintenance of consistent customer experience across different interaction modes **Results and Impact** The deployment achieved several key capabilities: * Successful greeting and customer assistance features * Navigation support across large product catalogs * Accurate product information delivery * Image-based product search and discovery * Maintenance of context through conversation flows The case study demonstrates several important LLMOps best practices: * Incremental deployment strategy starting with simpler features * Careful attention to data quality and augmentation * Custom adaptation of existing models (CLIP) for specific use cases * Balance between technical capability and practical business needs * Continuous evaluation and improvement based on user feedback **Ongoing Development** The team maintains an iterative development approach: * Regular updates based on customer feedback * Continuous research and development efforts * Planned expansions of capabilities * Focus on maintaining production stability while adding features This case study provides valuable insights into deploying complex AI systems in production e-commerce environments, highlighting the importance of balanced technical implementation with practical business needs. The incremental approach to feature deployment and careful attention to evaluation and monitoring demonstrate mature LLMOps practices.

Start your new ML Project today with ZenML Pro

Join 1,000s of members already deploying models with ZenML.