# Dropbox's Investigation of Control Character-Based Prompt Injection in LLMs ## Background and Context Dropbox has been exploring large language models (LLMs) for various product and research initiatives. As part of their security assessment for LLM deployments, they discovered a novel prompt injection technique using control characters that could circumvent system instructions in OpenAI's models. ## Technical Discovery ### Initial Setup - Dropbox used a standard prompt template for Q&A applications - Template included: ### Control Character Investigation - Team discovered unusual behavior with control characters in LLM inputs - Key findings about control characters: ## Technical Implementation Details ### Testing Methodology - Created systematic experiments using OpenAI's Chat API - Tested against both GPT-3.5 and GPT-4 models - Variables tested: ### Key Technical Findings - Model Behavior Changes: - Token Window Impact: ## Production Implementation Considerations ### Security Implications - Potential for: ### Mitigation Strategies - Input Sanitization: - Model Selection: ### Deployment Recommendations - Implement comprehensive input validation - Consider model-specific sanitization rules - Test thoroughly for prompt injection vulnerabilities - Balance security with legitimate control character uses - Regular security audits of prompt templates ## Development and Testing Practices ### Test Suite Development - Created Python test framework - Systematic testing of: ### Monitoring and Evaluation - Track model responses for: ## Future Considerations ### Ongoing Research Areas - Investigation of other control character combinations - Testing against different LLM providers - Development of comprehensive sanitization strategies - Evaluation of model-specific vulnerabilities ### Best Practices Evolution - Need for standardized security testing - Development of input sanitization libraries - Regular security assessments - Update procedures for new vulnerabilities ## Infrastructure Requirements ### API Integration - Proper handling of JSON payloads - Character encoding management - Request/response validation - Error handling procedures ### Security Controls - Input validation layers - Character encoding checks - Context window monitoring - Response validation ## Production Deployment Guidelines ### Implementation Checklist - Input sanitization mechanisms - Control character handling - Context verification - Response validation - Error handling - Security monitoring ### Risk Management - Regular security assessments - Prompt template audits - Model behavior monitoring - Input validation updates - Incident response procedures This case study represents significant research in LLM security and provides valuable insights for organizations deploying LLMs in production environments. The discoveries about control character behavior in prompt injection attacks highlight the importance of thorough security testing and proper input sanitization in LLM applications.