# Linguistic-Informed Approach to LLM Systems at Mastercard ## Overview Chris Brousseau, a lead data scientist at Mastercard, presents a comprehensive framework for implementing LLMs in production environments by focusing on linguistic principles rather than just traditional metrics. The approach emphasizes the importance of understanding and incorporating linguistic features to create more effective and maintainable LLM systems. ## Key Linguistic Components in LLM Systems ### Syntax - LLMs have largely solved syntactical challenges through transformational generative grammar - Models can generate infinite combinations of grammatically correct structures - Current implementations effectively handle basic grammar rules ### Morphology - Implemented through tokenization and embeddings - Current solutions are approximately 75-80% effective - Challenges with statistical tokenization methods affecting model performance - Issues highlighted with numerical operations in larger models like GPT-4 vs. smaller models like Goat-7B ### Semantics and Pragmatics - Current focus areas for improvement in LLM systems - Dictionary Problem: Need to handle evolving language definitions - Importance of regular vocabulary updates in production systems - Consideration of domain-specific language stability (e.g., financial terminology at Mastercard) ## Technical Implementation Challenges ### The Dictionary Problem - Dictionaries represent snapshots of language usage - Need for dynamic vocabulary updates in production systems - Weekly/monthly soft updates and yearly hard updates recommended - Special considerations for domain-specific applications - Balance between currency and stability in financial applications ### The Tokenization Challenge ("Yeet Problem") - Issues with popular tokenization methods like BPE and SentencePiece - Impact on arithmetic operations in large models - Importance of understanding allowable sound and letter combinations - Need for predictable tokenization patterns - Benefits of multilingual approaches in improving tokenization ### The Morphological Challenge ("Kimono Problem") - Issues with splitting borrowed words - Importance of understanding basic units of meaning - Benefits of multilingual models in handling diverse vocabularies - Need for context-aware tokenization ## Practical Implementation Example ### Biology Question-Answering System - Baseline: Vanilla ChatGPT implementation - Optimized Implementation: - Results: ## Production Considerations ### Model Maintenance - Regular updates to handle evolving language - Balance between model currency and stability - Domain-specific considerations for update frequency - Monitoring of vocabulary shifts and usage patterns ### Performance Optimization - Local deployment options for faster inference - Use of guidance framework for improved accuracy - Integration with other tools like LangChain - Vector databases for document retrieval ### Multimodal Considerations - Text-to-speech challenges - Speech-to-speech implementations - Phonetic information preservation - Integration of International Phonetic Alphabet ## Best Practices and Recommendations ### Development Approach - Focus on linguistic features rather than just metrics - Consider domain-specific language requirements - Implement regular update cycles - Use multilingual approaches when possible ### Tool Selection - Local deployment options for performance - Integration of linguistic frameworks - Use of Chain of Thought reasoning - Implementation of pragmatic instruction ### Monitoring and Maintenance - Regular vocabulary updates - Performance tracking - Accuracy measurements - Response time optimization ## Future Directions - Expansion of pragmatic instruction capabilities - Integration with document retrieval systems - Improvement of phonetic handling - Enhanced multilingual support - Development of more sophisticated update mechanisms