# Using LLMs to Scale Insurance Operations at Anzen ## Company Background Anzen is tackling the $23 billion problem of employee lawsuits against employers. They provide two main services: - Insurance coverage for businesses against employee lawsuits - Software platform for risk management and compliance - Small team of under 20 people competing with large insurance companies ## LLM Implementation #1: Automated Underwriting Process ### The Challenge - Traditional underwriting requires manual processing of complex insurance applications - Applications come in various formats, are long and dense - Need to process quickly for insurance brokers who work with multiple carriers - Small team needs to compete with larger insurance companies' resources ### Technical Solution - Two-part system implemented: ### Document Classification System - Built using Google's BERT model - Training data: - Performance metrics: - Tested multiple open-source models from Hugging Face - Optimized for high recall over precision due to use case requirements ### Information Extraction - Utilized AWS Textract's question-answering feature - Allows extraction of specific information through natural language queries - Likely powered by LLM technology under the hood - Implementation completed in under a week ## LLM Implementation #2: Compliance Document Review System ### The Challenge - Companies heavily rely on lawyers for document review - Need for immediate feedback on potential compliance issues - High accuracy requirements due to legal implications ### Technical Solution - Two-step process for document analysis: - Implementation details: - Prototype developed in approximately one week ## Key LLMOps Learnings ### Infrastructure Considerations - Resource-intensive models require careful infrastructure planning - Standard production system considerations become more critical - Smaller models can run on CPU instances for prototypes - Important to plan for scaling and high load scenarios ### Evaluation and Monitoring - Critical to establish quantitative metrics - Need to monitor performance over time - Important to compare test performance vs production performance ### Cost Considerations - API costs can be significant for iterative development - Need to factor in costs for: - Self-hosting vs API trade-offs need careful consideration ### Future Opportunities - Potential for using GPT models as underwriting assistants - Natural language interfaces for compliance information - Exploring ways to handle larger context windows - Considering summarization approaches for handling multiple documents ## Technical Architecture Notes - Uses combination of open-source and commercial APIs - Modular system design allowing for component updates - Balance between automation and human oversight - Focus on practical implementation over perfect accuracy ## Production Considerations - Non-deterministic nature of LLM outputs requires robust error handling - Need for continuous monitoring and evaluation - Important to have fallback mechanisms - Regular retraining and model updates may be necessary - Balance between model complexity and practical deployment needs